Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260923 17:30:54.300213 16346 test_util.cc:268] Using random seed: -664345692
2026-09-23T17:30:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T17:30:54Z Disabled control of system clock
I20260923 17:30:54.360817 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.190:33707
--webserver_interface=127.15.246.190
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260923 17:30:54.654560 16355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:30:54.655216 16355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:30:54.655680 16355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:30:54.686748 16355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:30:54.687073 16355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:30:54.687382 16355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:30:54.687862 16355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:30:54.724475 16355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.190:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.246.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:30:54.725912 16355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:30:54.727591 16355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:30:54.749212 16362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:30:54.749269 16361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:54.751282 16355 server_base.cc:1034] running on GCE node
W20260923 17:30:54.752717 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:55.863389 16355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:30:55.866896 16355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:30:55.868391 16355 hybrid_clock.cc:648] HybridClock initialized: now 1790184655868343 us; error 82 us; skew 500 ppm
I20260923 17:30:55.869288 16355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:30:55.876757 16355 webserver.cc:460] Webserver started at http://127.15.246.190:41135/ using document root <none> and password file <none>
I20260923 17:30:55.877769 16355 fs_manager.cc:359] Metadata directory not provided
I20260923 17:30:55.877970 16355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:30:55.878422 16355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:30:55.884562 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data/instance:
uuid: "1dbd0d3324414f26ab5d34c375921ef2"
format_stamp: "Formatted at 2026-09-23 17:30:55 on dist-test-slave-mk2q"
I20260923 17:30:55.885859 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal/instance:
uuid: "1dbd0d3324414f26ab5d34c375921ef2"
format_stamp: "Formatted at 2026-09-23 17:30:55 on dist-test-slave-mk2q"
I20260923 17:30:55.894624 16355 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.008s
I20260923 17:30:55.902000 16371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:30:55.903337 16355 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260923 17:30:55.903720 16355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal
uuid: "1dbd0d3324414f26ab5d34c375921ef2"
format_stamp: "Formatted at 2026-09-23 17:30:55 on dist-test-slave-mk2q"
I20260923 17:30:55.904083 16355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:30:55.962481 16355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:30:55.964102 16355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:30:55.964578 16355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:30:56.054100 16355 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.190:33707
I20260923 17:30:56.054207 16424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.190:33707 every 8 connection(s)
I20260923 17:30:56.057140 16355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
I20260923 17:30:56.061005 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16355
I20260923 17:30:56.061548 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-0/wal/instance
I20260923 17:30:56.064926 16425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:30:56.067028 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:45151
--webserver_interface=127.15.246.189
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:30:56.079453 16425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:56.115386 16425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:30:56.117897 16374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
W20260923 17:30:56.122308 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
I20260923 17:30:56.171900 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 1
W20260923 17:30:56.176012 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
I20260923 17:30:56.244475 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 2
W20260923 17:30:56.248660 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
I20260923 17:30:56.340092 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 3
W20260923 17:30:56.344210 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
W20260923 17:30:56.416654 16428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:30:56.417338 16428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:30:56.417804 16428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:30:56.450366 16428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:30:56.450719 16428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:30:56.451006 16428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:30:56.451251 16428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:30:56.487654 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 4
I20260923 17:30:56.488754 16428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.246.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:30:56.490983 16428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260923 17:30:56.492769 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
I20260923 17:30:56.493281 16428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:30:56.516170 16437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:56.792265 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 5
W20260923 17:30:56.796437 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
I20260923 17:30:57.343881 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 6
W20260923 17:30:57.352368 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:45151: Network error: Client connection negotiation failed: client connection to 127.15.246.189:45151: connect: Connection refused (error 111)
W20260923 17:30:56.516283 16440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:56.517889 16428 server_base.cc:1034] running on GCE node
W20260923 17:30:56.516209 16438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:57.635607 16428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:30:57.638988 16428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:30:57.640468 16428 hybrid_clock.cc:648] HybridClock initialized: now 1790184657640410 us; error 89 us; skew 500 ppm
I20260923 17:30:57.641322 16428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:30:57.658514 16428 webserver.cc:460] Webserver started at http://127.15.246.189:37925/ using document root <none> and password file <none>
I20260923 17:30:57.659590 16428 fs_manager.cc:359] Metadata directory not provided
I20260923 17:30:57.659792 16428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:30:57.660312 16428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:30:57.670502 16428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data/instance:
uuid: "70172383e8bf40a49974968e8a8f5b23"
format_stamp: "Formatted at 2026-09-23 17:30:57 on dist-test-slave-mk2q"
I20260923 17:30:57.672437 16428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal/instance:
uuid: "70172383e8bf40a49974968e8a8f5b23"
format_stamp: "Formatted at 2026-09-23 17:30:57 on dist-test-slave-mk2q"
I20260923 17:30:57.683228 16428 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260923 17:30:57.690239 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:30:57.691768 16428 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260923 17:30:57.692338 16428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal
uuid: "70172383e8bf40a49974968e8a8f5b23"
format_stamp: "Formatted at 2026-09-23 17:30:57 on dist-test-slave-mk2q"
I20260923 17:30:57.692873 16428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:30:57.733621 16428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:30:57.735368 16428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:30:57.735858 16428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:30:57.831363 16428 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.189:45151
I20260923 17:30:57.831440 16502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.189:45151 every 8 connection(s)
I20260923 17:30:57.834594 16428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
I20260923 17:30:57.841557 16503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:30:57.842660 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16428
I20260923 17:30:57.843078 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-1/wal/instance
I20260923 17:30:57.845543 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:46075
--webserver_interface=127.15.246.188
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:30:57.854692 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:57.891130 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:57.909621 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:30:57.912959 16452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
W20260923 17:30:57.918552 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:57.968184 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 1
W20260923 17:30:57.972786 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.041332 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 2
W20260923 17:30:58.045675 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.137158 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 3
W20260923 17:30:58.141963 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
W20260923 17:30:58.168671 16505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:30:58.169247 16505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:30:58.169678 16505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:30:58.200140 16505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:30:58.200531 16505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:30:58.200789 16505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:30:58.201030 16505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:30:58.239192 16505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.246.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:30:58.240469 16505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:30:58.242053 16505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:30:58.262061 16518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:58.285457 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 4
W20260923 17:30:58.290894 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.412786 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } attempt: 7
I20260923 17:30:58.428691 16425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:30:58.435395 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.493803 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 1
W20260923 17:30:58.497978 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.579386 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 2
W20260923 17:30:58.583588 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.590266 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 5
W20260923 17:30:58.593959 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.669010 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 3
W20260923 17:30:58.673063 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:58.813657 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 4
W20260923 17:30:58.820766 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:59.104378 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 5
W20260923 17:30:59.108203 16425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:30:59.141600 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 6
W20260923 17:30:59.148351 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:46075: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
W20260923 17:30:58.262219 16516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:59.405045 16505 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260923 17:30:59.404994 16517 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260923 17:30:58.263324 16515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:30:59.410130 16505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:30:59.414283 16505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:30:59.415771 16505 hybrid_clock.cc:648] HybridClock initialized: now 1790184659415733 us; error 56 us; skew 500 ppm
I20260923 17:30:59.416762 16505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:30:59.425024 16505 webserver.cc:460] Webserver started at http://127.15.246.188:33399/ using document root <none> and password file <none>
I20260923 17:30:59.426074 16505 fs_manager.cc:359] Metadata directory not provided
I20260923 17:30:59.426301 16505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:30:59.426797 16505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:30:59.432348 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/instance:
uuid: "012948ec54c1441381b43989690264ce"
format_stamp: "Formatted at 2026-09-23 17:30:59 on dist-test-slave-mk2q"
I20260923 17:30:59.433576 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal/instance:
uuid: "012948ec54c1441381b43989690264ce"
format_stamp: "Formatted at 2026-09-23 17:30:59 on dist-test-slave-mk2q"
I20260923 17:30:59.442691 16505 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260923 17:30:59.449086 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:30:59.450455 16505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 17:30:59.450822 16505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
uuid: "012948ec54c1441381b43989690264ce"
format_stamp: "Formatted at 2026-09-23 17:30:59 on dist-test-slave-mk2q"
I20260923 17:30:59.451179 16505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:30:59.484750 16505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:30:59.486388 16505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:30:59.486867 16505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:30:59.575961 16505 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.188:46075
I20260923 17:30:59.576058 16582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.188:46075 every 8 connection(s)
I20260923 17:30:59.578928 16505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
I20260923 17:30:59.585609 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16505
I20260923 17:30:59.586098 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal/instance
I20260923 17:30:59.586091 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:30:59.602218 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:59.630784 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:59.648887 16583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:30:59.660754 16425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 6
I20260923 17:30:59.674376 16583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Bootstrap starting.
I20260923 17:30:59.681700 16583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Neither blocks nor log segments found. Creating new log.
I20260923 17:30:59.684139 16425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Bootstrap starting.
I20260923 17:30:59.684412 16583 log.cc:824] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Log is configured to *not* fsync() on all Append() calls
I20260923 17:30:59.690670 16583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: No bootstrap required, opened a new log
I20260923 17:30:59.690765 16425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Neither blocks nor log segments found. Creating new log.
I20260923 17:30:59.693040 16425 log.cc:824] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Log is configured to *not* fsync() on all Append() calls
I20260923 17:30:59.698650 16425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: No bootstrap required, opened a new log
I20260923 17:30:59.721535 16583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.722424 16583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:30:59.722711 16583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012948ec54c1441381b43989690264ce, State: Initialized, Role: FOLLOWER
I20260923 17:30:59.723644 16583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.727406 16596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.728528 16596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:30:59.729280 16583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:30:59.731129 16425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.731961 16425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:30:59.732319 16425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbd0d3324414f26ab5d34c375921ef2, State: Initialized, Role: FOLLOWER
I20260923 17:30:59.733312 16425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.737105 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.738175 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:30:59.738982 16425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 17:30:59.743006 16536 tablet.cc:2369] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 17:30:59.770372 16613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:30:59.770859 16613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:30:59.784852 16620 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 17:30:59.764394 16505 master_runner.cc:186] Error getting master registration for 127.15.246.190:33707: OK, instance_id { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" instance_seqno: 1790184656029230 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 17:30:59.800282 16505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:30:59.791751 16355 master_runner.cc:186] Error getting master registration for 127.15.246.189:45151: OK, instance_id { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" instance_seqno: 1790184657805333 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 17:30:59.814399 16355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:30:59.826050 16600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:30:59.826723 16600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.829830 16600 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:30:59.832536 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:30:59.833554 16558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbd0d3324414f26ab5d34c375921ef2 in term 0.
I20260923 17:30:59.835094 16373 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2; no voters:
I20260923 17:30:59.836747 16600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:30:59.837685 16600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:30:59.838392 16600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:30:59.850005 16600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.854861 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce"
I20260923 17:30:59.855415 16558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:30:59.860883 16600 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 1 election: Requested vote from peers 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:30:59.862257 16558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbd0d3324414f26ab5d34c375921ef2 in term 1.
I20260923 17:30:59.863287 16373 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2; no voters:
I20260923 17:30:59.864605 16600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:30:59.866421 16600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 1dbd0d3324414f26ab5d34c375921ef2, State: Running, Role: LEADER
I20260923 17:30:59.867825 16600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:30:59.876101 16621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.882508 16621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:30:59.886193 16623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:30:59.897549 16623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 17:30:59.918641 16558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Refusing update from remote peer 1dbd0d3324414f26ab5d34c375921ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 17:30:59.920290 16600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:30:59.957856 16596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.959160 16596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:30:59.962406 16625 mvcc.cc:205] Tried to move back new op lower bound from 7332596367006322688 to 7332596366828105728. Current Snapshot: MvccSnapshot[applied={T|T < 7332596367006322688}]
I20260923 17:30:59.965230 16621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.966178 16621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:30:59.970463 16596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:30:59.971191 16596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:30:59.973673 16623 catalog_manager.cc:1347] Generated new cluster ID: b555dcd485a344bc8bb1e61ebbffcc17
I20260923 17:30:59.973953 16623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:31:00.018225 16623 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 17:31:00.021173 16623 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:31:00.050249 16623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Generated new TSK 0
I20260923 17:31:00.052592 16623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 17:31:00.082439 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.129:0
--local_ip_for_outbound_sockets=127.15.246.129
--webserver_interface=127.15.246.129
--webserver_port=0
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:31:00.208886 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } attempt: 7
I20260923 17:31:00.266891 16503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: Bootstrap starting.
I20260923 17:31:00.288434 16503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:00.291447 16503 log.cc:824] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:00.308704 16503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: No bootstrap required, opened a new log
I20260923 17:31:00.374545 16503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:00.375536 16503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:00.375838 16503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70172383e8bf40a49974968e8a8f5b23, State: Initialized, Role: FOLLOWER
I20260923 17:31:00.389240 16503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:00.409866 16635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:00.411460 16635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:00.412397 16635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:00.413249 16635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:00.415132 16503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:31:00.415542 16475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:00.416925 16475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbd0d3324414f26ab5d34c375921ef2 in term 0.
I20260923 17:31:00.418923 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:00.420866 16476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:00.423642 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:00.422371 16635 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:00.425228 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:00.415532 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23"
W20260923 17:31:00.428552 16477 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36778 (request call id 3) took 576 ms (client timeout 712 ms). Trace:
I20260923 17:31:00.421792 16478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
W20260923 17:31:00.430490 16478 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36778 (request call id 2) took 600 ms (client timeout 548 ms). Trace:
W20260923 17:31:00.429719 16477 rpcz_store.cc:269] 0923 17:30:59.852429 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:30:59.852568 (+ 139us) service_pool.cc:227] Handling call
0923 17:31:00.428483 (+575915us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260923 17:31:00.430961 16478 rpcz_store.cc:269] 0923 17:30:59.830299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:30:59.830441 (+ 142us) service_pool.cc:227] Handling call
0923 17:31:00.430423 (+599982us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:00.443025 16476 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Refusing update from remote peer 1dbd0d3324414f26ab5d34c375921ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 17:31:00.443602 16635 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:00.449654 16600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:00.494262 16640 mvcc.cc:205] Tried to move back new op lower bound from 7332596367006322688 to 7332596366828105728. Current Snapshot: MvccSnapshot[applied={T|T < 7332596367006322688}]
I20260923 17:31:00.514674 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:00.516722 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:00.515957 16635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:00.517630 16635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:00.591163 16653 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260923 17:31:00.698083 16630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:00.699013 16630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:00.699976 16630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:00.736056 16630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:00.737133 16630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.129
I20260923 17:31:00.776912 16613 catalog_manager.cc:1259] Loaded cluster ID: b555dcd485a344bc8bb1e61ebbffcc17
I20260923 17:31:00.777338 16613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: loading cluster ID for follower catalog manager: success
I20260923 17:31:00.780156 16630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.246.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.129
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:00.782193 16630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:00.784782 16630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 17:31:00.786417 16613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: acquiring CA information for follower catalog manager: success
I20260923 17:31:00.792337 16613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 17:31:00.823480 16657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:00.955590 16635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:00.956107 16635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:00.957963 16635 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:00.959460 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:00.959709 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:00.961498 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:00.962404 16635 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:01.071053 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:01.072028 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:31:01.588901 16664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:01.589365 16664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:01.592013 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:01.593969 16664 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:01.597116 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:01.599144 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:01.600035 16664 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:01.606679 16653 catalog_manager.cc:1259] Loaded cluster ID: b555dcd485a344bc8bb1e61ebbffcc17
I20260923 17:31:01.607084 16653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: loading cluster ID for follower catalog manager: success
I20260923 17:31:01.636937 16653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: acquiring CA information for follower catalog manager: success
I20260923 17:31:01.648087 16653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:31:01.917244 16666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:01.917954 16666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:01.921574 16666 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:01.921795 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:01.921795 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:01.923972 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:01.925313 16666 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:02.140278 16667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:02.141129 16667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:02.145150 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:02.147320 16667 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:02.148837 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:02.150569 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:02.156468 16667 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:02.206436 16656 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 16630
W20260923 17:31:02.386322 16630 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 1.566s user 0.521s sys 0.772s
W20260923 17:31:02.386684 16630 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 1.567s user 0.521s sys 0.773s
W20260923 17:31:00.824388 16658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:02.387523 16656 kernel_stack_watchdog.cc:198] Thread 16630 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 384ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260923 17:31:02.398758 16668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:02.400507 16630 server_base.cc:1034] running on GCE node
I20260923 17:31:02.402017 16630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:02.412297 16630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:02.413805 16630 hybrid_clock.cc:648] HybridClock initialized: now 1790184662413763 us; error 62 us; skew 500 ppm
I20260923 17:31:02.415134 16630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:02.436303 16630 webserver.cc:460] Webserver started at http://127.15.246.129:43611/ using document root <none> and password file <none>
I20260923 17:31:02.437777 16630 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:02.438071 16630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:02.438685 16630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:02.446234 16630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data/instance:
uuid: "be93fda0843b44f5abc49e6f35c69217"
format_stamp: "Formatted at 2026-09-23 17:31:02 on dist-test-slave-mk2q"
I20260923 17:31:02.447959 16630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/instance:
uuid: "be93fda0843b44f5abc49e6f35c69217"
format_stamp: "Formatted at 2026-09-23 17:31:02 on dist-test-slave-mk2q"
I20260923 17:31:02.473454 16674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:02.474228 16674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:02.477356 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:02.480459 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:02.482143 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:02.485502 16630 fs_manager.cc:691] Time spent creating directory manager: real 0.037s user 0.007s sys 0.007s
I20260923 17:31:02.491103 16674 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:02.492199 16674 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:02.504376 16676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:02.506283 16630 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.001s sys 0.009s
I20260923 17:31:02.506810 16630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal
uuid: "be93fda0843b44f5abc49e6f35c69217"
format_stamp: "Formatted at 2026-09-23 17:31:02 on dist-test-slave-mk2q"
I20260923 17:31:02.507449 16630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:02.590873 16630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:02.593111 16630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:02.593848 16630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:02.604323 16630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:31:02.616729 16630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:31:02.617170 16630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260923 17:31:02.617576 16630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:31:02.617879 16630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:02.705003 16688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:02.705669 16688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:02.707612 16688 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:02.708688 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:02.711360 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:02.713231 16454 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:02.720633 16688 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:03.064873 16757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:03.065436 16757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:03.068193 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:03.070837 16757 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:03.070952 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:03.072367 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:03.073345 16757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:03.201362 16630 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.129:38879
I20260923 17:31:03.201496 16791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.129:38879 every 8 connection(s)
I20260923 17:31:03.205749 16630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
I20260923 17:31:03.211282 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16630
I20260923 17:31:03.211746 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/instance
I20260923 17:31:03.225628 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.130:0
--local_ip_for_outbound_sockets=127.15.246.130
--webserver_interface=127.15.246.130
--webserver_port=0
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:31:03.298391 16801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:03.300922 16801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:03.307037 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:03.328377 16793 heartbeater.cc:347] Connected to a master server at 127.15.246.189:45151
I20260923 17:31:03.329101 16793 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:03.331332 16793 heartbeater.cc:511] Master 127.15.246.189:45151 requested a full tablet report, sending...
I20260923 17:31:03.342511 16792 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:03.343130 16792 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:03.344949 16792 heartbeater.cc:511] Master 127.15.246.188:46075 requested a full tablet report, sending...
I20260923 17:31:03.363219 16547 ts_manager.cc:194] Registered new tserver with Master: be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:03.364558 16801 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:03.366178 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:03.368031 16468 ts_manager.cc:194] Registered new tserver with Master: be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:03.368641 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:03.370476 16801 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:03.383245 16794 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:03.383781 16794 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:03.385537 16794 heartbeater.cc:511] Master 127.15.246.190:33707 requested a full tablet report, sending...
I20260923 17:31:03.390730 16388 ts_manager.cc:194] Registered new tserver with Master: be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:03.394925 16388 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.129:58511
I20260923 17:31:03.592504 16804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:03.593175 16804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:03.595661 16804 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:03.596647 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:03.597750 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:03.599959 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:03.601742 16804 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:03.825402 16799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:03.825924 16799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:03.826495 16799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:03.879473 16799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:03.880523 16799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.130
I20260923 17:31:03.901127 16794 heartbeater.cc:503] Master 127.15.246.190:33707 was elected leader, sending a full tablet report...
I20260923 17:31:03.916154 16799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.246.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.130
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:03.917461 16799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:03.919040 16799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:03.944592 16811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:04.252434 16816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:04.253207 16816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:04.255452 16816 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:04.256781 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:04.257048 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:04.259440 16454 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:04.260519 16816 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:04.848498 16819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:04.849144 16819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:04.852453 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:04.852957 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:04.854529 16819 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:04.854847 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:04.855870 16819 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:03.944977 16810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:03.946784 16813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:05.084431 16812 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260923 17:31:05.084605 16799 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260923 17:31:05.088738 16799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:05.091228 16799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:05.092595 16799 hybrid_clock.cc:648] HybridClock initialized: now 1790184665092541 us; error 71 us; skew 500 ppm
I20260923 17:31:05.093478 16799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:05.100399 16799 webserver.cc:460] Webserver started at http://127.15.246.130:41529/ using document root <none> and password file <none>
I20260923 17:31:05.101434 16799 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:05.101655 16799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:05.102162 16799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:05.107547 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data/instance:
uuid: "c36e79694e154f84a46980719217d7f2"
format_stamp: "Formatted at 2026-09-23 17:31:05 on dist-test-slave-mk2q"
I20260923 17:31:05.108866 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/instance:
uuid: "c36e79694e154f84a46980719217d7f2"
format_stamp: "Formatted at 2026-09-23 17:31:05 on dist-test-slave-mk2q"
I20260923 17:31:05.117173 16799 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260923 17:31:05.123622 16825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:05.124982 16799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260923 17:31:05.125371 16799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal
uuid: "c36e79694e154f84a46980719217d7f2"
format_stamp: "Formatted at 2026-09-23 17:31:05 on dist-test-slave-mk2q"
I20260923 17:31:05.125749 16799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:05.164335 16799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:05.165959 16799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:05.166411 16799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:05.169251 16799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:31:05.174422 16799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:31:05.174649 16799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:05.174925 16799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:31:05.175077 16799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:05.364894 16799 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.130:35433
I20260923 17:31:05.364964 16937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.130:35433 every 8 connection(s)
I20260923 17:31:05.367964 16799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
I20260923 17:31:05.378693 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16799
I20260923 17:31:05.379357 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/instance
I20260923 17:31:05.403095 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.131:0
--local_ip_for_outbound_sockets=127.15.246.131
--webserver_interface=127.15.246.131
--webserver_port=0
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:31:05.444283 16940 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:05.444792 16940 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:05.446026 16940 heartbeater.cc:511] Master 127.15.246.190:33707 requested a full tablet report, sending...
I20260923 17:31:05.448740 16388 ts_manager.cc:194] Registered new tserver with Master: c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:05.449183 16939 heartbeater.cc:347] Connected to a master server at 127.15.246.189:45151
I20260923 17:31:05.449389 16938 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:05.449599 16939 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:05.449697 16938 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:05.450481 16388 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.130:46845
I20260923 17:31:05.450663 16939 heartbeater.cc:511] Master 127.15.246.189:45151 requested a full tablet report, sending...
I20260923 17:31:05.451709 16938 heartbeater.cc:511] Master 127.15.246.188:46075 requested a full tablet report, sending...
I20260923 17:31:05.453338 16468 ts_manager.cc:194] Registered new tserver with Master: c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:05.454284 16546 ts_manager.cc:194] Registered new tserver with Master: c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:05.464483 16949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:05.464874 16949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:05.466256 16949 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:05.467406 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:05.467566 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:05.469967 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:05.470647 16949 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:05.775593 16948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:05.776170 16948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:05.776784 16948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:05.809904 16948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:05.810968 16948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.131
I20260923 17:31:05.857983 16948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.246.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.131
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:05.859462 16948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:05.861429 16948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:05.884807 16956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:05.956689 16940 heartbeater.cc:503] Master 127.15.246.190:33707 was elected leader, sending a full tablet report...
I20260923 17:31:06.245795 16961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:06.246222 16961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:06.247996 16961 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:06.249593 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:06.249672 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:06.251430 16534 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:06.252693 16961 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:06.784525 16963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:06.784919 16963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:06.786377 16963 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:06.787396 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:06.787840 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:06.789510 16534 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:06.790227 16963 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:06.882463 16964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:06.882939 16964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:06.884567 16964 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:06.885711 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:06.885792 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:06.886315 16558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70172383e8bf40a49974968e8a8f5b23 in term 1.
I20260923 17:31:06.887593 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 012948ec54c1441381b43989690264ce, 70172383e8bf40a49974968e8a8f5b23; no voters: 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:06.888545 16964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 17:31:06.888814 16964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:06.889072 16964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 1 FOLLOWER]: Advancing to term 2
I20260923 17:31:06.898945 16964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:06.902096 16964 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 election: Requested vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:06.902375 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2"
I20260923 17:31:06.903676 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce"
I20260923 17:31:06.905074 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:06.905951 16964 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 17:31:06.978303 16477 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 2 FOLLOWER]: Rejecting Update request from peer 1dbd0d3324414f26ab5d34c375921ef2 for earlier term 1. Current term is 2. Ops: []
I20260923 17:31:06.979588 16965 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260923 17:31:06.981338 16966 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 LEADER]: Stepping down as leader of term 1
I20260923 17:31:06.981611 16966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1dbd0d3324414f26ab5d34c375921ef2, State: Running, Role: LEADER
I20260923 17:31:06.982215 16966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:06.983210 16966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 1 FOLLOWER]: Advancing to term 2
W20260923 17:31:05.886142 16958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:05.885447 16955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:05.888188 16948 server_base.cc:1034] running on GCE node
I20260923 17:31:07.030184 16948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:07.032882 16948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:07.034328 16948 hybrid_clock.cc:648] HybridClock initialized: now 1790184667034260 us; error 63 us; skew 500 ppm
I20260923 17:31:07.035235 16948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:07.043049 16948 webserver.cc:460] Webserver started at http://127.15.246.131:40313/ using document root <none> and password file <none>
I20260923 17:31:07.044211 16948 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:07.044485 16948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:07.045006 16948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:07.050669 16948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data/instance:
uuid: "e5c071cdf65042a59cad226a15a5532c"
format_stamp: "Formatted at 2026-09-23 17:31:07 on dist-test-slave-mk2q"
I20260923 17:31:07.052024 16948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/instance:
uuid: "e5c071cdf65042a59cad226a15a5532c"
format_stamp: "Formatted at 2026-09-23 17:31:07 on dist-test-slave-mk2q"
I20260923 17:31:07.060904 16948 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260923 17:31:07.067880 16971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:07.069232 16948 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260923 17:31:07.069585 16948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal
uuid: "e5c071cdf65042a59cad226a15a5532c"
format_stamp: "Formatted at 2026-09-23 17:31:07 on dist-test-slave-mk2q"
I20260923 17:31:07.069934 16948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:07.112150 16948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:07.113867 16948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:07.114378 16948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:07.117172 16948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:31:07.121541 16948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:31:07.121819 16948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:07.122089 16948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:31:07.122241 16948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:07.293644 16948 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.131:42559
I20260923 17:31:07.293810 17085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.131:42559 every 8 connection(s)
I20260923 17:31:07.296741 16948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
I20260923 17:31:07.298779 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 16948
I20260923 17:31:07.299289 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/instance
I20260923 17:31:07.360376 17086 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:07.361016 17086 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:07.362129 17086 heartbeater.cc:511] Master 127.15.246.188:46075 requested a full tablet report, sending...
I20260923 17:31:07.362644 17088 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:07.363092 17088 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:07.364334 17088 heartbeater.cc:511] Master 127.15.246.190:33707 requested a full tablet report, sending...
I20260923 17:31:07.364956 16966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:07.365381 16966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:07.366741 16546 ts_manager.cc:194] Registered new tserver with Master: e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:07.367228 16966 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:07.368649 16387 ts_manager.cc:194] Registered new tserver with Master: e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:07.368609 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:07.369105 16477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbd0d3324414f26ab5d34c375921ef2 in term 2.
I20260923 17:31:07.370291 16374 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23; no voters:
I20260923 17:31:07.370657 17087 heartbeater.cc:347] Connected to a master server at 127.15.246.189:45151
I20260923 17:31:07.370970 17087 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:07.371801 17087 heartbeater.cc:511] Master 127.15.246.189:45151 requested a full tablet report, sending...
I20260923 17:31:07.373179 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce" is_pre_election: true
I20260923 17:31:07.374574 16468 ts_manager.cc:194] Registered new tserver with Master: e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:07.375875 16966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 17:31:07.376256 16966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:07.376561 16966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 2 FOLLOWER]: Advancing to term 3
I20260923 17:31:07.379544 16346 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 17:31:07.387241 16966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:07.389003 16966 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 3 election: Requested vote from peers 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:07.390262 16558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "012948ec54c1441381b43989690264ce"
I20260923 17:31:07.390751 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dbd0d3324414f26ab5d34c375921ef2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23"
I20260923 17:31:07.391316 16477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 2 FOLLOWER]: Advancing to term 3
I20260923 17:31:07.400396 16477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbd0d3324414f26ab5d34c375921ef2 in term 3.
I20260923 17:31:07.405262 16374 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce
I20260923 17:31:07.407392 17097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:07.408000 17097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:07.408582 16966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 17:31:07.409369 16966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [term 3 LEADER]: Becoming Leader. State: Replica: 1dbd0d3324414f26ab5d34c375921ef2, State: Running, Role: LEADER
I20260923 17:31:07.410111 16966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:07.412446 17097 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:07.413671 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:07.414595 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:07.415271 16477 raft_consensus.cc:2354] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 012948ec54c1441381b43989690264ce for earlier term 2. Current term is 3.
I20260923 17:31:07.416617 16533 leader_election.cc:398] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Vote denied by peer 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 012948ec54c1441381b43989690264ce for earlier term 2. Current term is 3.
I20260923 17:31:07.416982 16533 leader_election.cc:401] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260923 17:31:07.417902 17097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 1 FOLLOWER]: Advancing to term 3
I20260923 17:31:07.425350 17097 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 012948ec54c1441381b43989690264ce for earlier term 2. Current term is 3.
I20260923 17:31:07.428587 16965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:07.429361 16965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:31:07.441797 17105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:31:07.448510 17105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 17:31:07.451864 17105 catalog_manager.cc:1259] Loaded cluster ID: b555dcd485a344bc8bb1e61ebbffcc17
I20260923 17:31:07.452080 17105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:31:07.455989 17105 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:31:07.460338 17105 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Loaded TSK: 0
I20260923 17:31:07.461548 17105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 17:31:07.489527 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_caded27fb8cc41fe9dd686fbd0e0e2ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:07.489084 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b491404014cd47fda02a4c2c6df07c18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:07.489078 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b56bcda5a5bb4e80b7876e41de9ed964"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:07.494117 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_60782fde812447b193d28b78f1e9d26d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:07.495397 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b56bcda5a5bb4e80b7876e41de9ed964 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:07.496616 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60782fde812447b193d28b78f1e9d26d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:07.494696 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b491404014cd47fda02a4c2c6df07c18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:07.506681 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_caded27fb8cc41fe9dd686fbd0e0e2ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:07.526568 16558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Refusing update from remote peer 1dbd0d3324414f26ab5d34c375921ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260923 17:31:07.527290 16477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Refusing update from remote peer 1dbd0d3324414f26ab5d34c375921ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260923 17:31:07.528803 16966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260923 17:31:07.530632 16965 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260923 17:31:07.549548 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16505
I20260923 17:31:07.556126 17125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:07.556845 17125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:07.561239 16966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:07.562000 16966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:31:07.567222 17125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:07.567905 17125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:07.591825 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_caded27fb8cc41fe9dd686fbd0e0e2ad" } new_table_name: "default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7" modify_external_catalogs: true
I20260923 17:31:07.596195 16386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_60782fde812447b193d28b78f1e9d26d" } modify_external_catalogs: true
W20260923 17:31:07.596542 16373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.246.188:46075 (error 108) [suppressed 13 similar messages]
I20260923 17:31:07.596858 16386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_60782fde812447b193d28b78f1e9d26d" } modify_external_catalogs: true
I20260923 17:31:07.598572 16387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b491404014cd47fda02a4c2c6df07c18" } new_table_name: "default.Table_3388e92db4d143039362fc01aafe0a8b" modify_external_catalogs: true
I20260923 17:31:07.601675 16385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b56bcda5a5bb4e80b7876e41de9ed964" } modify_external_catalogs: true
I20260923 17:31:07.602196 16385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b56bcda5a5bb4e80b7876e41de9ed964" } modify_external_catalogs: true
W20260923 17:31:07.620954 17135 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 17:31:07.617069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.246.188:46075 (local address 127.0.0.1:52460)
0923 17:31:07.619357 (+ 2288us) negotiation.cc:107] Waiting for socket to connect
0923 17:31:07.619628 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":1802,"thread_start_us":1811,"threads_started":1}
W20260923 17:31:07.628940 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:07.652141 16386 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f10e89a2c9364900a0d77d32afd68c43
I20260923 17:31:07.655833 16385 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet eb299489c3154275a2b4f101c4e64f6d
W20260923 17:31:07.657227 16387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad21604508864f81b6e6303ed9f2b1cf consensus state has no leader
I20260923 17:31:07.651859 16873 tablet_service.cc:1467] Processing CreateTablet for tablet ad21604508864f81b6e6303ed9f2b1cf (DEFAULT_TABLE table=default.Table_b491404014cd47fda02a4c2c6df07c18 [id=977770180e1447249a7ad3bef66d71ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.661952 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad21604508864f81b6e6303ed9f2b1cf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:07.664904 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e7a0597be204878b400a6a696c98f88 has no consensus state
I20260923 17:31:07.676538 16726 tablet_service.cc:1467] Processing CreateTablet for tablet ad21604508864f81b6e6303ed9f2b1cf (DEFAULT_TABLE table=default.Table_b491404014cd47fda02a4c2c6df07c18 [id=977770180e1447249a7ad3bef66d71ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.678821 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad21604508864f81b6e6303ed9f2b1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.685750 17021 tablet_service.cc:1467] Processing CreateTablet for tablet ad21604508864f81b6e6303ed9f2b1cf (DEFAULT_TABLE table=default.Table_b491404014cd47fda02a4c2c6df07c18 [id=977770180e1447249a7ad3bef66d71ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.688195 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad21604508864f81b6e6303ed9f2b1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.695415 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a0597be204878b400a6a696c98f88 (DEFAULT_TABLE table=default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7 [id=80ffb936cc8d499cbc9f194f1b38993e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.696715 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a0597be204878b400a6a696c98f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.697659 16872 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a0597be204878b400a6a696c98f88 (DEFAULT_TABLE table=default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7 [id=80ffb936cc8d499cbc9f194f1b38993e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.699048 16872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a0597be204878b400a6a696c98f88. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:07.699947 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7 [id=80ffb936cc8d499cbc9f194f1b38993e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0e7a0597be204878b400a6a696c98f88 consensus state has no leader
I20260923 17:31:07.699968 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a0597be204878b400a6a696c98f88 (DEFAULT_TABLE table=default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7 [id=80ffb936cc8d499cbc9f194f1b38993e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.701222 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a0597be204878b400a6a696c98f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.724408 17143 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:07.735615 17143 tablet_bootstrap.cc:654] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:07.738605 17143 log.cc:824] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:07.754575 17143 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:07.755362 17143 ts_tablet_manager.cc:1397] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.016s
I20260923 17:31:07.774631 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_745ed5dc2e06444ca3a30c0630398962"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:07.777640 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_745ed5dc2e06444ca3a30c0630398962 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:07.794113 17144 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:07.803727 17143 raft_consensus.cc:348] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:07.805764 17144 tablet_bootstrap.cc:654] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:07.805754 17143 raft_consensus.cc:374] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:07.805909 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_0a84d7335c3347b68ec95fcfed639095"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:07.806383 17143 raft_consensus.cc:729] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:07.807819 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a84d7335c3347b68ec95fcfed639095 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:07.807821 17143 consensus_queue.cc:260] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:07.817236 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_10e0dfaa6ca946cc91f7c7b6dd04d539"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:07.818217 17144 log.cc:824] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Log is configured to *not* fsync() on all Append() calls
W20260923 17:31:07.819371 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10e0dfaa6ca946cc91f7c7b6dd04d539 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:07.820569 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ba9a40693fbc40cda7c43b3b9a5405f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:07.828809 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:07.861094 16388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3388e92db4d143039362fc01aafe0a8b" } modify_external_catalogs: true
I20260923 17:31:07.861827 17146 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:07.878557 17144 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:07.879228 17144 ts_tablet_manager.cc:1397] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.086s user 0.015s sys 0.012s
I20260923 17:31:07.871302 16384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_745ed5dc2e06444ca3a30c0630398962" } modify_external_catalogs: true
I20260923 17:31:07.873356 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_01f12c2af8c64b5d90c6e7e9cf0746a7" } new_table_name: "default.Table_94d1ed48e79d45eb868e8231febfc208" modify_external_catalogs: true
I20260923 17:31:07.884286 16384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_745ed5dc2e06444ca3a30c0630398962" } modify_external_catalogs: true
I20260923 17:31:07.893196 16388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3388e92db4d143039362fc01aafe0a8b" } modify_external_catalogs: true
I20260923 17:31:07.893690 16383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.131:54045
I20260923 17:31:07.916333 17144 raft_consensus.cc:348] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:07.917629 17144 raft_consensus.cc:374] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:07.917989 17144 raft_consensus.cc:729] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:07.919137 17144 consensus_queue.cc:260] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:07.925091 17143 ts_tablet_manager.cc:1428] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.169s user 0.050s sys 0.037s
I20260923 17:31:07.926038 16385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_10e0dfaa6ca946cc91f7c7b6dd04d539" } new_table_name: "default.Table_83cf1a10788644d0ab0d432f9fb5501a" modify_external_catalogs: true
W20260923 17:31:07.935985 16792 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:46075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
W20260923 17:31:07.935506 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e7a0597be204878b400a6a696c98f88 consensus state has no leader
W20260923 17:31:07.927023 16945 tablet.cc:2369] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 17:31:07.940006 16938 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:46075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:31:07.945274 17143 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:07.953406 17086 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:46075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:31:07.956933 17146 tablet_bootstrap.cc:654] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:07.961505 17143 tablet_bootstrap.cc:654] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:07.965885 17146 log.cc:824] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:07.973307 16381 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet ad21604508864f81b6e6303ed9f2b1cf (table default.Table_3388e92db4d143039362fc01aafe0a8b [id=977770180e1447249a7ad3bef66d71ad]). Expected version 1 got 0
I20260923 17:31:07.980387 17146 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:07.981272 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 (DEFAULT_TABLE table=default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.982046 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 (DEFAULT_TABLE table=default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.982581 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e1aa0b01b84ed091ff35792f7d4449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.983541 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e1aa0b01b84ed091ff35792f7d4449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.985114 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 (DEFAULT_TABLE table=default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:07.986263 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e1aa0b01b84ed091ff35792f7d4449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:07.995972 17143 tablet_bootstrap.cc:492] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:07.996538 17143 ts_tablet_manager.cc:1397] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.051s user 0.014s sys 0.003s
I20260923 17:31:07.999562 17143 raft_consensus.cc:348] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.000401 17143 raft_consensus.cc:374] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.000715 17143 raft_consensus.cc:729] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.001426 17143 consensus_queue.cc:260] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.009328 17146 ts_tablet_manager.cc:1397] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.149s user 0.013s sys 0.008s
I20260923 17:31:08.028941 17143 ts_tablet_manager.cc:1428] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20260923 17:31:08.032994 17143 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:08.039858 17143 tablet_bootstrap.cc:654] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.055569 17143 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:08.056075 17143 ts_tablet_manager.cc:1397] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.002s
I20260923 17:31:08.064343 17143 raft_consensus.cc:348] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.065220 17143 raft_consensus.cc:374] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.065507 17143 raft_consensus.cc:729] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.066342 17143 consensus_queue.cc:260] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.080358 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3cc0a05ecc974d7cbd185095f39fee2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.082198 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cc0a05ecc974d7cbd185095f39fee2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.084911 17143 ts_tablet_manager.cc:1428] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
W20260923 17:31:08.084964 16385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b38675078f8404db169774e32520b71 has no consensus state
I20260923 17:31:08.076656 17146 raft_consensus.cc:348] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.088722 17146 raft_consensus.cc:374] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.089250 17146 raft_consensus.cc:729] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.090375 16794 heartbeater.cc:503] Master 127.15.246.190:33707 was elected leader, sending a full tablet report...
I20260923 17:31:08.090447 17146 consensus_queue.cc:260] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.098960 17144 ts_tablet_manager.cc:1428] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.219s user 0.081s sys 0.035s
I20260923 17:31:08.104821 17144 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.105551 17088 heartbeater.cc:503] Master 127.15.246.190:33707 was elected leader, sending a full tablet report...
I20260923 17:31:08.108711 17146 ts_tablet_manager.cc:1428] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.099s user 0.037s sys 0.013s
W20260923 17:31:08.114008 16800 tablet.cc:2369] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:31:08.116158 17146 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:08.121964 17144 tablet_bootstrap.cc:654] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.122728 17146 tablet_bootstrap.cc:654] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.131789 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_31a5ccd1a3a343578865f236f24762a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.133766 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31a5ccd1a3a343578865f236f24762a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.141798 17144 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:08.142285 17144 ts_tablet_manager.cc:1397] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.007s
I20260923 17:31:08.141880 16383 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_0a84d7335c3347b68ec95fcfed639095" } new_table_name: "default.Table_3707e7badee14016abf918e82796b952" modify_external_catalogs: true
I20260923 17:31:08.145197 17144 raft_consensus.cc:348] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.145944 17144 raft_consensus.cc:374] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.146209 17144 raft_consensus.cc:729] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.146986 17144 consensus_queue.cc:260] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.155591 17144 ts_tablet_manager.cc:1428] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260923 17:31:08.163815 17144 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.177884 17144 tablet_bootstrap.cc:654] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.215123 17144 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:08.215615 17144 ts_tablet_manager.cc:1397] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.000s
I20260923 17:31:08.219519 17144 raft_consensus.cc:348] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.220289 17144 raft_consensus.cc:374] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.220559 17144 raft_consensus.cc:729] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.221241 17144 consensus_queue.cc:260] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.226210 17147 raft_consensus.cc:482] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:08.226890 17147 raft_consensus.cc:504] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.230020 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c2e743852b974423b0f78affcc17342f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.232121 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2e743852b974423b0f78affcc17342f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.236018 17144 ts_tablet_manager.cc:1428] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.020s user 0.003s sys 0.002s
I20260923 17:31:08.247478 17147 leader_election.cc:288] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:08.258430 16383 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf43276ac2e54b508722d899ce8cfe6e has no consensus state
I20260923 17:31:08.265031 17146 tablet_bootstrap.cc:492] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:08.269759 17146 ts_tablet_manager.cc:1397] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.154s user 0.043s sys 0.049s
I20260923 17:31:08.273993 17146 raft_consensus.cc:348] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.283818 17146 raft_consensus.cc:374] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.284327 17146 raft_consensus.cc:729] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.288952 17146 consensus_queue.cc:260] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.292078 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 6b38675078f8404db169774e32520b71 (DEFAULT_TABLE table=default.Table_83cf1a10788644d0ab0d432f9fb5501a [id=74563e8c9eaa480f86c62f576d025643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.293331 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b38675078f8404db169774e32520b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.294003 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:08.296712 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:08.298648 16384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:08.300858 16388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.307502 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:08.309353 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet ad21604508864f81b6e6303ed9f2b1cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:08.295833 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 6b38675078f8404db169774e32520b71 (DEFAULT_TABLE table=default.Table_83cf1a10788644d0ab0d432f9fb5501a [id=74563e8c9eaa480f86c62f576d025643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.317169 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b38675078f8404db169774e32520b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.303725 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:08.304535 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet ad21604508864f81b6e6303ed9f2b1cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:08.328722 17147 raft_consensus.cc:482] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:08.329684 17163 tablet_replica.cc:330] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:08.329295 17147 raft_consensus.cc:504] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.330752 17163 raft_consensus.cc:2229] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:08.331373 17163 raft_consensus.cc:2258] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:08.331367 17147 leader_election.cc:288] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:08.331868 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet ad21604508864f81b6e6303ed9f2b1cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:08.332535 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 6b38675078f8404db169774e32520b71 (DEFAULT_TABLE table=default.Table_83cf1a10788644d0ab0d432f9fb5501a [id=74563e8c9eaa480f86c62f576d025643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.333621 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b38675078f8404db169774e32520b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.336488 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:08.338094 16381 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:08.338127 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad21604508864f81b6e6303ed9f2b1cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:08.339231 16746 raft_consensus.cc:2454] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:08.340821 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e1aa0b01b84ed091ff35792f7d4449" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:08.340303 16383 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_31a5ccd1a3a343578865f236f24762a9" } new_table_name: "default.Table_00de54e4eb1a42e388cdb06cdc068f18" modify_external_catalogs: true
I20260923 17:31:08.342898 17163 ts_tablet_manager.cc:1905] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:08.343501 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_7f2fe719a90d41cd9d77b8be004f3d64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.345535 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f2fe719a90d41cd9d77b8be004f3d64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.346027 17144 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.352172 17144 tablet_bootstrap.cc:654] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:08.355372 17094 tablet.cc:2369] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:31:08.354440 16826 leader_election.cc:302] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:08.356928 17147 raft_consensus.cc:2781] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:08.357316 17147 raft_consensus.cc:482] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:08.357635 17147 raft_consensus.cc:3037] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:08.360566 17156 raft_consensus.cc:482] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:08.361205 17156 raft_consensus.cc:504] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.363679 17165 tablet_replica.cc:330] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:08.367923 16826 leader_election.cc:341] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:08.364403 17156 leader_election.cc:288] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:08.373489 17147 raft_consensus.cc:504] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.374840 17165 raft_consensus.cc:2229] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:08.386303 17144 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:08.386780 17144 ts_tablet_manager.cc:1397] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.000s
I20260923 17:31:08.389581 17144 raft_consensus.cc:348] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.390309 17144 raft_consensus.cc:374] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.390566 17144 raft_consensus.cc:729] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.391171 17144 consensus_queue.cc:260] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.391674 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a0597be204878b400a6a696c98f88" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:08.392560 16745 raft_consensus.cc:2454] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:08.394001 16972 leader_election.cc:302] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:08.395032 17156 raft_consensus.cc:2781] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:08.395411 17156 raft_consensus.cc:482] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:08.395749 17156 raft_consensus.cc:3037] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:08.405072 17152 raft_consensus.cc:482] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:08.405607 17152 raft_consensus.cc:504] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.411700 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_a81cb8a73c834d5189c93746ed9be7a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.413648 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a81cb8a73c834d5189c93746ed9be7a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.419952 17152 leader_election.cc:288] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:08.422076 17147 leader_election.cc:288] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:08.422809 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad21604508864f81b6e6303ed9f2b1cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:08.423393 16745 raft_consensus.cc:3037] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:08.429694 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_7021a8a1a2154d319a16208cfbc49dee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:08.438153 16745 raft_consensus.cc:2454] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:08.439512 16826 leader_election.cc:302] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:08.440411 17147 raft_consensus.cc:2684] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Received election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260923 17:31:08.441998 17144 ts_tablet_manager.cc:1428] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.055s user 0.006s sys 0.002s
I20260923 17:31:08.443778 17165 raft_consensus.cc:2258] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:08.449728 17165 ts_tablet_manager.cc:1905] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:08.439775 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7021a8a1a2154d319a16208cfbc49dee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:08.460718 16383 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a340a854bc54ac189a833e8de1bb8cf has no consensus state
I20260923 17:31:08.462468 17163 ts_tablet_manager.cc:1918] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:08.462918 17163 log.cc:1196] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:08.463920 17163 ts_tablet_manager.cc:1939] T 45e1aa0b01b84ed091ff35792f7d4449 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:08.467165 17163 tablet_replica.cc:330] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:08.467314 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 45e1aa0b01b84ed091ff35792f7d4449 (table default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]) successfully deleted
I20260923 17:31:08.467897 17163 raft_consensus.cc:2229] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:08.468374 17163 raft_consensus.cc:2258] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:08.470593 17163 ts_tablet_manager.cc:1905] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:08.471769 16383 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_83cf1a10788644d0ab0d432f9fb5501a" } new_table_name: "default.Table_7215630316334465a5446b982ca07e7e" modify_external_catalogs: true
I20260923 17:31:08.486397 17163 ts_tablet_manager.cc:1918] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:08.486811 17163 log.cc:1196] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.487474 17163 ts_tablet_manager.cc:1939] T ad21604508864f81b6e6303ed9f2b1cf P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:08.407912 17156 raft_consensus.cc:504] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.490506 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet ad21604508864f81b6e6303ed9f2b1cf (table default.Table_3388e92db4d143039362fc01aafe0a8b [id=977770180e1447249a7ad3bef66d71ad]) successfully deleted
I20260923 17:31:08.510445 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a0597be204878b400a6a696c98f88" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:08.510921 17156 leader_election.cc:288] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:08.511107 16745 raft_consensus.cc:3037] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:08.518390 16745 raft_consensus.cc:2454] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:08.519857 16972 leader_election.cc:302] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:08.524616 17167 raft_consensus.cc:2781] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:08.525228 17167 raft_consensus.cc:686] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:08.526331 16385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3707e7badee14016abf918e82796b952" } modify_external_catalogs: true
I20260923 17:31:08.526957 16385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3707e7badee14016abf918e82796b952" } modify_external_catalogs: true
I20260923 17:31:08.531581 16387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c2e743852b974423b0f78affcc17342f" } modify_external_catalogs: true
I20260923 17:31:08.532163 16387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c2e743852b974423b0f78affcc17342f" } modify_external_catalogs: true
I20260923 17:31:08.534492 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:08.537766 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:08.542900 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf (DEFAULT_TABLE table=default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.544130 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a340a854bc54ac189a833e8de1bb8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.551991 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a0597be204878b400a6a696c98f88" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:08.553423 16893 raft_consensus.cc:2454] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:08.554760 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet ad21604508864f81b6e6303ed9f2b1cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:08.556236 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:08.556594 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet ad21604508864f81b6e6303ed9f2b1cf because the tablet was not found. No further retry: Not found: Tablet not found: ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.556535 17143 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:08.558331 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 45e1aa0b01b84ed091ff35792f7d4449 because the tablet was not found. No further retry: Not found: Tablet not found: 45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:08.561055 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (DEFAULT_TABLE table=default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.562155 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:08.569046 16383 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b38675078f8404db169774e32520b71 consensus state has no leader
I20260923 17:31:08.570333 16724 tablet_service.cc:1467] Processing CreateTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 (DEFAULT_TABLE table=default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.571590 16724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86f4b7ee14948f0989af54b3b77dc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.576612 17144 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.577821 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6b38675078f8404db169774e32520b71 (table default.Table_7215630316334465a5446b982ca07e7e [id=74563e8c9eaa480f86c62f576d025643]). Expected version 2 got 0
I20260923 17:31:08.581384 16723 tablet_service.cc:1467] Processing CreateTablet for tablet bf43276ac2e54b508722d899ce8cfe6e (DEFAULT_TABLE table=default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.582656 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf43276ac2e54b508722d899ce8cfe6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.586895 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:08.590391 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a0597be204878b400a6a696c98f88" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:08.591320 16892 raft_consensus.cc:3037] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:08.591928 16677 leader_election.cc:341] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:08.594907 17144 tablet_bootstrap.cc:654] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.594650 16872 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (DEFAULT_TABLE table=default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.595913 16872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.599123 16870 tablet_service.cc:1467] Processing CreateTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 (DEFAULT_TABLE table=default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.600833 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86f4b7ee14948f0989af54b3b77dc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.586963 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf (DEFAULT_TABLE table=default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.605460 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a340a854bc54ac189a833e8de1bb8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.540594 17167 consensus_queue.cc:237] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.613353 17146 ts_tablet_manager.cc:1428] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.342s user 0.009s sys 0.004s
I20260923 17:31:08.624015 16869 tablet_service.cc:1467] Processing CreateTablet for tablet bf43276ac2e54b508722d899ce8cfe6e (DEFAULT_TABLE table=default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.626050 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf43276ac2e54b508722d899ce8cfe6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.616117 17156 raft_consensus.cc:482] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:08.632933 17156 raft_consensus.cc:504] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:08.635057 17156 leader_election.cc:288] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:08.639884 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad21604508864f81b6e6303ed9f2b1cf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:08.641059 17164 tablet_replica.cc:330] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
W20260923 17:31:08.641573 16972 leader_election.cc:341] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:31:08.642105 17146 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:08.645383 17143 tablet_bootstrap.cc:654] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:08.646003 16972 leader_election.cc:341] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.646483 16972 leader_election.cc:302] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:08.658975 17156 raft_consensus.cc:2726] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:08.660501 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e1aa0b01b84ed091ff35792f7d4449" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:08.662714 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad21604508864f81b6e6303ed9f2b1cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:08.663425 17039 raft_consensus.cc:3037] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:08.665162 16386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_7f2fe719a90d41cd9d77b8be004f3d64" } new_table_name: "default.Table_392d5ef452e04a2e9013847c2f2e6948" modify_external_catalogs: true
I20260923 17:31:08.668776 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad21604508864f81b6e6303ed9f2b1cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:08.674175 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_fcda1b375f0848edbbbc4ed3bdd00f79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:08.674998 16387 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 20f6f7c2e6f647f897331db0ec8ca7fa
W20260923 17:31:08.678157 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fcda1b375f0848edbbbc4ed3bdd00f79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.691089 17039 raft_consensus.cc:2454] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:08.695578 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c6d68da075ce4f84a7305de3ff3489f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.697602 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6d68da075ce4f84a7305de3ff3489f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:08.702455 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 45e1aa0b01b84ed091ff35792f7d4449 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45e1aa0b01b84ed091ff35792f7d4449 already in progress: creating tablet
I20260923 17:31:08.702847 17164 raft_consensus.cc:2229] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:08.705062 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf (DEFAULT_TABLE table=default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.693740 17146 tablet_bootstrap.cc:654] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.695794 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:08.706269 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a340a854bc54ac189a833e8de1bb8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.706012 17018 tablet_service.cc:1467] Processing CreateTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 (DEFAULT_TABLE table=default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:08.706987 16678 leader_election.cc:341] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:08.707491 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86f4b7ee14948f0989af54b3b77dc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.707441 16678 leader_election.cc:302] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:08.708401 17152 raft_consensus.cc:2726] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:08.711385 16373 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6b38675078f8404db169774e32520b71 (table default.Table_7215630316334465a5446b982ca07e7e [id=74563e8c9eaa480f86c62f576d025643], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6b38675078f8404db169774e32520b71 consensus state has no leader
I20260923 17:31:08.711213 17017 tablet_service.cc:1467] Processing CreateTablet for tablet bf43276ac2e54b508722d899ce8cfe6e (DEFAULT_TABLE table=default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.712901 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf43276ac2e54b508722d899ce8cfe6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:08.712772 16827 leader_election.cc:341] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:08.713256 16827 leader_election.cc:302] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:08.714978 17147 raft_consensus.cc:2726] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:08.716202 17164 raft_consensus.cc:2258] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:08.713805 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (DEFAULT_TABLE table=default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:08.718889 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:08.724560 16892 raft_consensus.cc:2454] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:08.736800 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c64deefc885140dd972d0106708b6281"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.738878 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c64deefc885140dd972d0106708b6281 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.741876 17164 ts_tablet_manager.cc:1905] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:08.757160 16385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet bf43276ac2e54b508722d899ce8cfe6e
I20260923 17:31:08.764138 17144 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:08.764662 17144 ts_tablet_manager.cc:1397] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.188s user 0.042s sys 0.045s
I20260923 17:31:08.766937 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:08.769678 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet bf43276ac2e54b508722d899ce8cfe6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf43276ac2e54b508722d899ce8cfe6e already in progress: creating tablet
I20260923 17:31:08.769873 17144 raft_consensus.cc:348] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:08.770592 17144 raft_consensus.cc:374] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.770840 17144 raft_consensus.cc:729] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.771457 17144 consensus_queue.cc:260] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:08.780539 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_7215630316334465a5446b982ca07e7e" } new_table_name: "default.Table_3f641ab8052a4e93a7feec1a9d3d9562" modify_external_catalogs: true
I20260923 17:31:08.786139 17144 ts_tablet_manager.cc:1428] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260923 17:31:08.792454 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:08.792613 17144 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.796146 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:08.799696 17144 tablet_bootstrap.cc:654] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.814072 17146 tablet_bootstrap.cc:492] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:08.814582 17146 ts_tablet_manager.cc:1397] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.173s user 0.007s sys 0.005s
I20260923 17:31:08.820799 17144 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:08.821224 17144 ts_tablet_manager.cc:1397] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.008s
I20260923 17:31:08.821585 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ab2253f3624c44c79de7adc058ba16bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:08.823803 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab2253f3624c44c79de7adc058ba16bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:08.823561 17144 raft_consensus.cc:348] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:08.827116 17146 raft_consensus.cc:348] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.832334 17144 raft_consensus.cc:374] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.832675 17144 raft_consensus.cc:729] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.833293 17144 consensus_queue.cc:260] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:08.827847 17146 raft_consensus.cc:374] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.834647 17146 raft_consensus.cc:729] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.835896 17144 ts_tablet_manager.cc:1428] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.014s user 0.002s sys 0.002s
I20260923 17:31:08.835507 17146 consensus_queue.cc:260] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:08.840500 16386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3d473eff8734b6286c5162df107c3e6 has no consensus state
I20260923 17:31:08.840692 17144 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:08.856649 17146 ts_tablet_manager.cc:1428] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.042s user 0.002s sys 0.004s
I20260923 17:31:08.857514 17144 tablet_bootstrap.cc:654] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.857642 17164 ts_tablet_manager.cc:1918] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:08.858261 17164 log.cc:1196] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.867395 17164 ts_tablet_manager.cc:1939] T ad21604508864f81b6e6303ed9f2b1cf P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:08.871384 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet ad21604508864f81b6e6303ed9f2b1cf (table default.Table_3388e92db4d143039362fc01aafe0a8b [id=977770180e1447249a7ad3bef66d71ad]) successfully deleted
I20260923 17:31:08.876739 17143 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:08.878587 17143 ts_tablet_manager.cc:1397] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.326s user 0.012s sys 0.003s
I20260923 17:31:08.879956 17146 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:08.881142 17143 raft_consensus.cc:348] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.881922 17143 raft_consensus.cc:374] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.882192 17143 raft_consensus.cc:729] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.882956 17143 consensus_queue.cc:260] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:08.885607 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet bf43276ac2e54b508722d899ce8cfe6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf43276ac2e54b508722d899ce8cfe6e already in progress: creating tablet
I20260923 17:31:08.896433 16380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_94d1ed48e79d45eb868e8231febfc208" } modify_external_catalogs: true
I20260923 17:31:08.897027 16380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_94d1ed48e79d45eb868e8231febfc208" } modify_external_catalogs: true
I20260923 17:31:08.897423 17146 tablet_bootstrap.cc:654] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.915294 17146 tablet_bootstrap.cc:492] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:08.916038 17146 ts_tablet_manager.cc:1397] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.010s
I20260923 17:31:08.920154 17146 raft_consensus.cc:348] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.921034 17146 raft_consensus.cc:374] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:08.921317 17146 raft_consensus.cc:729] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:08.932252 17146 consensus_queue.cc:260] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:08.945438 17146 ts_tablet_manager.cc:1428] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260923 17:31:08.946429 17165 ts_tablet_manager.cc:1918] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:08.946936 17165 log.cc:1196] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:08.948163 17165 ts_tablet_manager.cc:1939] T ad21604508864f81b6e6303ed9f2b1cf P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:08.956894 17146 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:08.964709 17146 tablet_bootstrap.cc:654] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:08.953148 17143 ts_tablet_manager.cc:1428] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.074s user 0.006s sys 0.001s
I20260923 17:31:08.972697 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet ad21604508864f81b6e6303ed9f2b1cf (table default.Table_3388e92db4d143039362fc01aafe0a8b [id=977770180e1447249a7ad3bef66d71ad]) successfully deleted
I20260923 17:31:08.973675 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_392d5ef452e04a2e9013847c2f2e6948" } modify_external_catalogs: true
I20260923 17:31:08.974295 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_392d5ef452e04a2e9013847c2f2e6948" } modify_external_catalogs: true
I20260923 17:31:08.975956 17165 tablet_replica.cc:330] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:08.976680 17165 raft_consensus.cc:2229] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:08.977141 17165 raft_consensus.cc:2258] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:08.979347 17165 ts_tablet_manager.cc:1905] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:08.980877 17143 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:09.003073 17146 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.003545 17146 ts_tablet_manager.cc:1397] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.008s
I20260923 17:31:09.004451 17143 tablet_bootstrap.cc:654] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.006217 17146 raft_consensus.cc:348] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.006935 17146 raft_consensus.cc:374] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.007205 17146 raft_consensus.cc:729] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.007853 17146 consensus_queue.cc:260] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.018069 17146 ts_tablet_manager.cc:1428] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.014s user 0.009s sys 0.000s
I20260923 17:31:09.021788 17146 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.024019 17017 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8154a3aed44924b7127bd4c8452c19 (DEFAULT_TABLE table=default.Table_a81cb8a73c834d5189c93746ed9be7a7 [id=d5acb2052772421a8369fefb1e359fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.025132 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8154a3aed44924b7127bd4c8452c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.025483 16385 catalog_manager.cc:5495] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:09.027560 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]). Expected version 2 got 0
I20260923 17:31:09.028944 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a340a854bc54ac189a833e8de1bb8cf (table default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]). Expected version 1 got 0
I20260923 17:31:09.030284 17021 tablet_service.cc:1467] Processing CreateTablet for tablet f3d473eff8734b6286c5162df107c3e6 (DEFAULT_TABLE table=default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.031390 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d473eff8734b6286c5162df107c3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.031689 17146 tablet_bootstrap.cc:654] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.035133 17165 ts_tablet_manager.cc:1918] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.035517 17165 log.cc:1196] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:09.035467 16724 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8154a3aed44924b7127bd4c8452c19 (DEFAULT_TABLE table=default.Table_a81cb8a73c834d5189c93746ed9be7a7 [id=d5acb2052772421a8369fefb1e359fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.036131 17165 ts_tablet_manager.cc:1939] T 45e1aa0b01b84ed091ff35792f7d4449 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:09.036191 16387 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6b38675078f8404db169774e32520b71 (table default.Table_7215630316334465a5446b982ca07e7e [id=74563e8c9eaa480f86c62f576d025643]). Expected version 2 got 0
I20260923 17:31:09.036590 16724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8154a3aed44924b7127bd4c8452c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.040544 16723 tablet_service.cc:1467] Processing CreateTablet for tablet f3d473eff8734b6286c5162df107c3e6 (DEFAULT_TABLE table=default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.040879 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 45e1aa0b01b84ed091ff35792f7d4449 (table default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]) successfully deleted
I20260923 17:31:09.041707 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d473eff8734b6286c5162df107c3e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:09.043236 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b38675078f8404db169774e32520b71 consensus state has no leader
W20260923 17:31:09.055147 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet bf43276ac2e54b508722d899ce8cfe6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf43276ac2e54b508722d899ce8cfe6e already in progress: creating tablet
I20260923 17:31:09.058259 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6b38675078f8404db169774e32520b71 (table default.Table_3f641ab8052a4e93a7feec1a9d3d9562 [id=74563e8c9eaa480f86c62f576d025643]). Expected version 3 got 0
I20260923 17:31:09.061048 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c6d68da075ce4f84a7305de3ff3489f1" } new_table_name: "default.Table_162de7ee3faf4ec3916e0cd2bcef3d45" modify_external_catalogs: true
I20260923 17:31:09.072206 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a340a854bc54ac189a833e8de1bb8cf (table default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]). Expected version 1 got 0
I20260923 17:31:09.109366 16870 tablet_service.cc:1467] Processing CreateTablet for tablet f3d473eff8734b6286c5162df107c3e6 (DEFAULT_TABLE table=default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.111179 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d473eff8734b6286c5162df107c3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.123826 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c9193016785148b492ccdc68f136e29b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:09.125830 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9193016785148b492ccdc68f136e29b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.126386 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8154a3aed44924b7127bd4c8452c19 (DEFAULT_TABLE table=default.Table_a81cb8a73c834d5189c93746ed9be7a7 [id=d5acb2052772421a8369fefb1e359fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.127615 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8154a3aed44924b7127bd4c8452c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.138183 17144 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:09.138630 17144 ts_tablet_manager.cc:1397] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.298s user 0.055s sys 0.049s
I20260923 17:31:09.141656 17144 raft_consensus.cc:348] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.142351 17144 raft_consensus.cc:374] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.142606 17144 raft_consensus.cc:729] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.143345 17144 consensus_queue.cc:260] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.145442 17144 ts_tablet_manager.cc:1428] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260923 17:31:09.146847 17144 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:09.150548 17146 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.151503 17144 tablet_bootstrap.cc:654] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.151239 17146 ts_tablet_manager.cc:1397] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.130s user 0.032s sys 0.026s
I20260923 17:31:09.153234 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ab80ba4ac749415083b0ebac2b5401e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:09.155550 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab80ba4ac749415083b0ebac2b5401e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.157843 17146 raft_consensus.cc:348] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.158576 17146 raft_consensus.cc:374] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.159804 17144 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:09.160192 17144 ts_tablet_manager.cc:1397] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260923 17:31:09.162662 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:09.162707 17144 raft_consensus.cc:348] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.163292 17144 raft_consensus.cc:374] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.163511 17144 raft_consensus.cc:729] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.158840 17146 raft_consensus.cc:729] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.164016 17144 consensus_queue.cc:260] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.164922 17146 consensus_queue.cc:260] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.172484 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:09.174091 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f3d473eff8734b6286c5162df107c3e6 because the tablet was not found. No further retry: Not found: Tablet not found: f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:09.174705 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.175261 17152 raft_consensus.cc:482] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.175362 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.176254 17152 raft_consensus.cc:504] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.177240 17163 tablet_replica.cc:330] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:09.178031 17163 raft_consensus.cc:2229] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.178505 17163 raft_consensus.cc:2258] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.179766 17152 leader_election.cc:288] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.179752 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:09.181267 17144 ts_tablet_manager.cc:1428] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260923 17:31:09.181411 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:09.182088 17144 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:09.183166 16677 leader_election.cc:341] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.184182 17152 raft_consensus.cc:482] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.184577 17152 raft_consensus.cc:504] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
W20260923 17:31:09.185000 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet f3d473eff8734b6286c5162df107c3e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3d473eff8734b6286c5162df107c3e6 already in progress: creating tablet
W20260923 17:31:09.185645 16678 leader_election.cc:341] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.187853 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:09.189149 16677 leader_election.cc:341] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.189668 17152 leader_election.cc:288] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.196409 17167 raft_consensus.cc:482] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.196943 17167 raft_consensus.cc:504] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.198881 17167 leader_election.cc:288] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.201357 17144 tablet_bootstrap.cc:654] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.201315 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:09.203007 16972 leader_election.cc:341] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.204655 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:09.205324 16745 raft_consensus.cc:2454] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:09.206642 16972 leader_election.cc:302] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:09.209210 17167 raft_consensus.cc:2781] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:09.209563 17167 raft_consensus.cc:482] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:09.209833 17167 raft_consensus.cc:3037] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.212800 17163 ts_tablet_manager.cc:1905] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.241523 17167 raft_consensus.cc:504] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.245172 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:09.245838 16745 raft_consensus.cc:3037] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.191641 16678 leader_election.cc:302] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:09.252894 17152 raft_consensus.cc:2726] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:09.253031 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 69770.7us
I20260923 17:31:09.253822 16745 raft_consensus.cc:2454] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:09.255430 16972 leader_election.cc:302] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:09.257046 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:09.257936 17039 raft_consensus.cc:2379] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:09.259881 16678 leader_election.cc:302] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:09.179770 17146 ts_tablet_manager.cc:1428] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.026s user 0.002s sys 0.005s
I20260923 17:31:09.261595 17146 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:09.264781 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 102775us
I20260923 17:31:09.266630 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.270401 17143 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.270855 17143 ts_tablet_manager.cc:1397] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.290s user 0.044s sys 0.088s
I20260923 17:31:09.270853 17167 leader_election.cc:288] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.271449 17167 raft_consensus.cc:2781] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
W20260923 17:31:09.272267 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f3d473eff8734b6286c5162df107c3e6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3d473eff8734b6286c5162df107c3e6 already in progress: creating tablet
I20260923 17:31:09.273269 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_fcda1b375f0848edbbbc4ed3bdd00f79" } new_table_name: "default.Table_7ef3741399e44b2fa037d322957f7deb" modify_external_catalogs: true
I20260923 17:31:09.273763 17143 raft_consensus.cc:348] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.274497 17143 raft_consensus.cc:374] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.274770 17143 raft_consensus.cc:729] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.276503 17143 consensus_queue.cc:260] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.280113 17143 ts_tablet_manager.cc:1428] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260923 17:31:09.282496 17143 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:09.271868 17167 raft_consensus.cc:686] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
W20260923 17:31:09.292565 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21cf533908d04a3a8ca087355528556f has no consensus state
I20260923 17:31:09.296274 17143 tablet_bootstrap.cc:654] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:09.302124 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 46786.8us
I20260923 17:31:09.289278 17167 consensus_queue.cc:237] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.304550 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:09.309523 16972 leader_election.cc:341] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.315186 17147 raft_consensus.cc:482] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.315662 17147 raft_consensus.cc:504] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.317728 17147 leader_election.cc:288] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:09.320029 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
W20260923 17:31:09.322350 16827 leader_election.cc:341] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.329185 17143 tablet_bootstrap.cc:492] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.329697 17143 ts_tablet_manager.cc:1397] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.047s user 0.017s sys 0.009s
I20260923 17:31:09.310398 17144 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:09.331848 17144 ts_tablet_manager.cc:1397] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.150s user 0.014s sys 0.040s
I20260923 17:31:09.332594 17143 raft_consensus.cc:348] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.333630 17143 raft_consensus.cc:374] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.333933 17143 raft_consensus.cc:729] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.334503 17144 raft_consensus.cc:348] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.335157 17144 raft_consensus.cc:374] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.334733 17143 consensus_queue.cc:260] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.335390 17144 raft_consensus.cc:729] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.319160 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:09.336041 16745 raft_consensus.cc:2454] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:09.337198 17144 consensus_queue.cc:260] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.339313 17144 ts_tablet_manager.cc:1428] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20260923 17:31:09.339191 16826 leader_election.cc:302] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:09.339944 17147 raft_consensus.cc:2781] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:09.340253 17147 raft_consensus.cc:482] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:09.340545 17147 raft_consensus.cc:3037] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.342011 17143 ts_tablet_manager.cc:1428] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260923 17:31:09.344905 17144 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:09.346050 17143 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:09.351531 17147 raft_consensus.cc:504] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.352437 17143 tablet_bootstrap.cc:654] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.354305 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:09.356769 17152 raft_consensus.cc:2726] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:09.357653 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 97912.6us
W20260923 17:31:09.359066 16827 leader_election.cc:341] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:09.362334 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:09.362936 16745 raft_consensus.cc:3037] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.377629 16745 raft_consensus.cc:2454] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:09.379014 16826 leader_election.cc:302] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:09.396587 17147 leader_election.cc:288] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:09.400528 17147 raft_consensus.cc:2781] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:09.407393 17147 raft_consensus.cc:686] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:09.409401 17163 ts_tablet_manager.cc:1918] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.410290 17163 log.cc:1196] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/bf43276ac2e54b508722d899ce8cfe6e
I20260923 17:31:09.411258 17163 ts_tablet_manager.cc:1939] T bf43276ac2e54b508722d899ce8cfe6e P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:09.413008 17144 tablet_bootstrap.cc:654] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.414860 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet bf43276ac2e54b508722d899ce8cfe6e (table default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]) successfully deleted
I20260923 17:31:09.417238 17147 consensus_queue.cc:237] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.420440 17197 raft_consensus.cc:482] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.422937 17167 raft_consensus.cc:482] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.423424 17167 raft_consensus.cc:504] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:09.424976 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_d7bb20f8a5544384ab1e69311e05400e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:09.425348 17167 leader_election.cc:288] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.420985 17197 raft_consensus.cc:504] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
W20260923 17:31:09.426625 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7bb20f8a5544384ab1e69311e05400e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.428977 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.429100 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:09.429435 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:09.429735 16745 raft_consensus.cc:2454] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:09.429659 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b8377c11b1cc4acebc14e04381f6c57b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:09.430092 16892 raft_consensus.cc:2454] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:09.430727 17163 tablet_replica.cc:330] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:09.431393 17163 raft_consensus.cc:2229] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.431703 17163 raft_consensus.cc:2258] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.432350 16972 leader_election.cc:302] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:09.433209 17167 raft_consensus.cc:2781] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:09.433528 17167 raft_consensus.cc:482] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:09.433802 17167 raft_consensus.cc:3037] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.436499 17163 ts_tablet_manager.cc:1905] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:09.432066 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8377c11b1cc4acebc14e04381f6c57b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:09.441104 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 62149.3us
I20260923 17:31:09.443253 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:09.444072 16745 raft_consensus.cc:2379] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:09.447384 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a340a854bc54ac189a833e8de1bb8cf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:09.396077 17146 tablet_bootstrap.cc:654] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.452694 16827 leader_election.cc:302] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:09.453933 16380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 0e7a0597be204878b400a6a696c98f88
W20260923 17:31:09.454131 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a4ca6aa0faa244799949fd44c5db72ed has no consensus state
I20260923 17:31:09.459182 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a0597be204878b400a6a696c98f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.459932 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet ad21604508864f81b6e6303ed9f2b1cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.460609 17164 tablet_replica.cc:330] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:09.449996 17167 raft_consensus.cc:504] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:09.461340 17164 raft_consensus.cc:2229] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:09.462108 17164 pending_rounds.cc:62] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:09.462313 17164 pending_rounds.cc:68] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596402662604800 op_type: NO_OP noop_request { }
I20260923 17:31:09.462767 17164 raft_consensus.cc:2866] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:09.463091 17164 raft_consensus.cc:2258] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.463796 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:09.464462 16892 raft_consensus.cc:3037] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.465010 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a0597be204878b400a6a696c98f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.470641 17164 ts_tablet_manager.cc:1905] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.471460 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b38675078f8404db169774e32520b71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:09.472013 16745 raft_consensus.cc:3037] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.474550 16892 raft_consensus.cc:2454] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:09.476567 16972 leader_election.cc:302] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:09.477404 17191 raft_consensus.cc:2781] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:09.483997 17167 leader_election.cc:288] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:09.485697 16380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_00de54e4eb1a42e388cdb06cdc068f18" } modify_external_catalogs: true
I20260923 17:31:09.486459 16380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_00de54e4eb1a42e388cdb06cdc068f18" } modify_external_catalogs: true
I20260923 17:31:09.488386 17196 raft_consensus.cc:3037] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.492913 17146 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.492918 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_793e622592644e9daae1b9dba1524dba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:09.493388 17146 ts_tablet_manager.cc:1397] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.232s user 0.004s sys 0.010s
W20260923 17:31:09.494536 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_793e622592644e9daae1b9dba1524dba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.496662 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_7021a8a1a2154d319a16208cfbc49dee" } new_table_name: "default.Table_2b330ec3646e4cecac1e58f4d2c2d79a" modify_external_catalogs: true
I20260923 17:31:09.497711 17191 raft_consensus.cc:686] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:09.498715 17191 consensus_queue.cc:237] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:09.499419 17164 ts_tablet_manager.cc:1918] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:09.500560 17164 log.cc:1196] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/0e7a0597be204878b400a6a696c98f88
I20260923 17:31:09.501312 17143 tablet_bootstrap.cc:492] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.501794 17143 ts_tablet_manager.cc:1397] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.156s user 0.015s sys 0.021s
I20260923 17:31:09.502560 17164 ts_tablet_manager.cc:1939] T 0e7a0597be204878b400a6a696c98f88 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:09.504856 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_d0e18da1609f485e9684fb7a0228fc35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:09.505579 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]) successfully deleted
W20260923 17:31:09.507500 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0e18da1609f485e9684fb7a0228fc35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.507706 16745 raft_consensus.cc:2454] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:09.502779 17197 leader_election.cc:288] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:09.512127 17143 raft_consensus.cc:348] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.514756 17144 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:09.515090 17143 raft_consensus.cc:374] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.515511 17143 raft_consensus.cc:729] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.516539 17144 ts_tablet_manager.cc:1397] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.172s user 0.007s sys 0.006s
I20260923 17:31:09.506399 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a0597be204878b400a6a696c98f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:09.517340 17165 tablet_replica.cc:330] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:09.518105 17165 raft_consensus.cc:2229] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.495914 17146 raft_consensus.cc:348] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.518555 17165 raft_consensus.cc:2258] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.518886 17146 raft_consensus.cc:374] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.519169 17146 raft_consensus.cc:729] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:09.519992 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet ad21604508864f81b6e6303ed9f2b1cf because the tablet was not found. No further retry: Not found: Tablet not found: ad21604508864f81b6e6303ed9f2b1cf
I20260923 17:31:09.521519 17165 ts_tablet_manager.cc:1905] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.528514 17146 consensus_queue.cc:260] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.530493 17146 ts_tablet_manager.cc:1428] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.037s user 0.003s sys 0.002s
I20260923 17:31:09.531263 17146 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.531198 17143 consensus_queue.cc:260] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.536499 17146 tablet_bootstrap.cc:654] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.538625 17144 raft_consensus.cc:348] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.539744 17144 raft_consensus.cc:374] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.541674 17143 ts_tablet_manager.cc:1428] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.032s user 0.005s sys 0.001s
I20260923 17:31:09.542986 17143 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:09.551765 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63808fe254ec4b1f818ed3198d5aea3d has no consensus state
I20260923 17:31:09.540050 17144 raft_consensus.cc:729] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.553766 17144 consensus_queue.cc:260] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.566185 17144 ts_tablet_manager.cc:1428] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260923 17:31:09.569131 17191 raft_consensus.cc:482] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.569720 17191 raft_consensus.cc:504] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:09.573298 17191 leader_election.cc:288] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
W20260923 17:31:09.575851 16972 leader_election.cc:341] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Not found: Tablet not found: 45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:09.578823 17146 tablet_bootstrap.cc:492] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.579282 17146 ts_tablet_manager.cc:1397] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.048s user 0.016s sys 0.023s
I20260923 17:31:09.581818 17146 raft_consensus.cc:348] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.582777 17146 raft_consensus.cc:374] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.583182 17146 raft_consensus.cc:729] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.583878 17146 consensus_queue.cc:260] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
W20260923 17:31:09.586381 16972 leader_election.cc:341] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: 45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:09.586738 17146 ts_tablet_manager.cc:1428] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.007s user 0.010s sys 0.000s
I20260923 17:31:09.586854 16972 leader_election.cc:302] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:09.587587 17146 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.588524 17201 raft_consensus.cc:2726] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:09.588168 17165 ts_tablet_manager.cc:1918] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.563858 17163 ts_tablet_manager.cc:1918] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.591564 17163 log.cc:1196] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:09.592796 17146 tablet_bootstrap.cc:654] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.594107 17163 ts_tablet_manager.cc:1939] T f3d473eff8734b6286c5162df107c3e6 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:09.606998 17146 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.607417 17146 ts_tablet_manager.cc:1397] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.002s
I20260923 17:31:09.605419 17143 tablet_bootstrap.cc:654] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.607818 17165 log.cc:1196] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/0e7a0597be204878b400a6a696c98f88
I20260923 17:31:09.610293 17165 ts_tablet_manager.cc:1939] T 0e7a0597be204878b400a6a696c98f88 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:09.606760 17196 raft_consensus.cc:2726] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:09.613968 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]) successfully deleted
I20260923 17:31:09.620923 17163 tablet_replica.cc:330] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:09.622083 17163 raft_consensus.cc:2229] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.622246 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet f3d473eff8734b6286c5162df107c3e6 (table default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]) successfully deleted
I20260923 17:31:09.623212 17163 raft_consensus.cc:2258] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.625291 17146 raft_consensus.cc:348] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.626431 17146 raft_consensus.cc:374] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.626704 17146 raft_consensus.cc:729] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.627354 17146 consensus_queue.cc:260] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.629799 17146 ts_tablet_manager.cc:1428] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.020s user 0.003s sys 0.003s
I20260923 17:31:09.630628 17146 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.636288 17163 ts_tablet_manager.cc:1905] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.636806 17146 tablet_bootstrap.cc:654] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.646749 17143 tablet_bootstrap.cc:492] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.647312 17143 ts_tablet_manager.cc:1397] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.104s user 0.014s sys 0.014s
I20260923 17:31:09.650720 17143 raft_consensus.cc:348] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.651473 17143 raft_consensus.cc:374] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.651767 17143 raft_consensus.cc:729] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.652431 17143 consensus_queue.cc:260] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.654397 17143 ts_tablet_manager.cc:1428] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260923 17:31:09.655813 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6b38675078f8404db169774e32520b71 (table default.Table_3f641ab8052a4e93a7feec1a9d3d9562 [id=74563e8c9eaa480f86c62f576d025643]). Expected version 3 got 0
I20260923 17:31:09.662992 16382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3f641ab8052a4e93a7feec1a9d3d9562" } modify_external_catalogs: true
I20260923 17:31:09.663255 17143 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:09.662424 16381 catalog_manager.cc:5495] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:09.663520 16382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3f641ab8052a4e93a7feec1a9d3d9562" } modify_external_catalogs: true
I20260923 17:31:09.665441 16380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 6a340a854bc54ac189a833e8de1bb8cf
I20260923 17:31:09.671613 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.672438 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:09.673292 17165 tablet_replica.cc:330] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:09.673293 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a340a854bc54ac189a833e8de1bb8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.674062 17164 tablet_replica.cc:330] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:09.674096 17165 raft_consensus.cc:2229] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.674597 17165 raft_consensus.cc:2258] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.674872 17164 raft_consensus.cc:2229] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:09.675591 17164 pending_rounds.cc:62] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:09.675817 17164 pending_rounds.cc:68] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596405478256640 op_type: NO_OP noop_request { }
I20260923 17:31:09.678586 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ab2253f3624c44c79de7adc058ba16bb" } new_table_name: "default.Table_6a55fbb756f44eb0800509b3e21e2134" modify_external_catalogs: true
I20260923 17:31:09.680507 17164 raft_consensus.cc:2866] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:09.681052 17164 raft_consensus.cc:2258] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.678198 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.678694 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e1aa0b01b84ed091ff35792f7d4449 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:07 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.682988 17165 ts_tablet_manager.cc:1905] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.692621 16380 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_4c7c906814484e6e95b4066c4ce97fb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:09.694463 16380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c7c906814484e6e95b4066c4ce97fb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.694703 17143 tablet_bootstrap.cc:654] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.700151 17164 ts_tablet_manager.cc:1905] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.702914 17146 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.703415 17146 ts_tablet_manager.cc:1397] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.073s user 0.022s sys 0.013s
I20260923 17:31:09.706120 17146 raft_consensus.cc:348] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.707185 17146 raft_consensus.cc:374] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.707491 17146 raft_consensus.cc:729] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.708241 17146 consensus_queue.cc:260] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.712699 17163 ts_tablet_manager.cc:1918] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.713137 17163 log.cc:1196] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/0e7a0597be204878b400a6a696c98f88
I20260923 17:31:09.715144 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 0721fa178c0347598351b9d3512089b8 (DEFAULT_TABLE table=default.Table_ab2253f3624c44c79de7adc058ba16bb [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.716879 17143 tablet_bootstrap.cc:492] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.717303 17143 ts_tablet_manager.cc:1397] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.000s
I20260923 17:31:09.718331 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0721fa178c0347598351b9d3512089b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.719488 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_e9b25f5740c5469ab07a12b480144735"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:09.719674 17143 raft_consensus.cc:348] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.720386 17143 raft_consensus.cc:374] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.720602 17143 raft_consensus.cc:729] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:09.721077 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9b25f5740c5469ab07a12b480144735 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.721915 17163 ts_tablet_manager.cc:1939] T 0e7a0597be204878b400a6a696c98f88 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:09.721297 17143 consensus_queue.cc:260] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.727492 17143 ts_tablet_manager.cc:1428] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260923 17:31:09.727857 17146 ts_tablet_manager.cc:1428] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.024s user 0.003s sys 0.003s
I20260923 17:31:09.730078 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 0721fa178c0347598351b9d3512089b8 (DEFAULT_TABLE table=default.Table_ab2253f3624c44c79de7adc058ba16bb [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.731199 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0721fa178c0347598351b9d3512089b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.733196 16724 tablet_service.cc:1467] Processing CreateTablet for tablet 0721fa178c0347598351b9d3512089b8 (DEFAULT_TABLE table=default.Table_ab2253f3624c44c79de7adc058ba16bb [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.734309 16724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0721fa178c0347598351b9d3512089b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:09.739285 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0721fa178c0347598351b9d3512089b8 consensus state has no leader
I20260923 17:31:09.743439 17017 tablet_service.cc:1467] Processing CreateTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d (DEFAULT_TABLE table=default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.744593 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63808fe254ec4b1f818ed3198d5aea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.751400 17165 ts_tablet_manager.cc:1918] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.751837 17165 log.cc:1196] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/6a340a854bc54ac189a833e8de1bb8cf
I20260923 17:31:09.752393 16382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 6b38675078f8404db169774e32520b71
I20260923 17:31:09.755041 17163 tablet_replica.cc:330] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:09.755615 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b38675078f8404db169774e32520b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:09.758564 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b38675078f8404db169774e32520b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.759140 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:09.759608 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 0e7a0597be204878b400a6a696c98f88 (table default.Table_94d1ed48e79d45eb868e8231febfc208 [id=80ffb936cc8d499cbc9f194f1b38993e]) successfully deleted
W20260923 17:31:09.762020 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63808fe254ec4b1f818ed3198d5aea3d consensus state has no leader
I20260923 17:31:09.761893 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_203b4089c5ed4f77ba3eb3f0230ac163"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:09.763840 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_203b4089c5ed4f77ba3eb3f0230ac163 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.764889 17143 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:09.764775 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_7ef3741399e44b2fa037d322957f7deb" } new_table_name: "default.Table_2c9fdbd8c49e4d9dbe68c5157000786d" modify_external_catalogs: true
I20260923 17:31:09.775751 17146 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.756515 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b38675078f8404db169774e32520b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:09.775945 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet bf43276ac2e54b508722d899ce8cfe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:09.777406 17163 raft_consensus.cc:2229] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.778040 17163 raft_consensus.cc:2258] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.786545 17165 ts_tablet_manager.cc:1939] T 6a340a854bc54ac189a833e8de1bb8cf P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:09.794415 17163 ts_tablet_manager.cc:1905] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.795866 17143 tablet_bootstrap.cc:654] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.796859 17146 tablet_bootstrap.cc:654] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.799130 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d (DEFAULT_TABLE table=default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.800412 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63808fe254ec4b1f818ed3198d5aea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.804798 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d473eff8734b6286c5162df107c3e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:08 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:09.805343 17164 ts_tablet_manager.cc:1918] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:09.805723 17164 log.cc:1196] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/6a340a854bc54ac189a833e8de1bb8cf
I20260923 17:31:09.806526 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d (DEFAULT_TABLE table=default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.808759 17144 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:09.809724 17165 tablet_replica.cc:330] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:09.810109 16872 tablet_service.cc:1467] Processing CreateTablet for tablet a4ca6aa0faa244799949fd44c5db72ed (DEFAULT_TABLE table=default.Table_7ef3741399e44b2fa037d322957f7deb [id=d0fff78552fb4121923c53b1a33d9994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.810510 17165 raft_consensus.cc:2229] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.812814 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 21fc7428f55041b6b939a8ca08574b90 (DEFAULT_TABLE table=default.Table_c64deefc885140dd972d0106708b6281 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.813699 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63808fe254ec4b1f818ed3198d5aea3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.813663 17164 ts_tablet_manager.cc:1939] T 6a340a854bc54ac189a833e8de1bb8cf P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
W20260923 17:31:09.820057 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a4ca6aa0faa244799949fd44c5db72ed consensus state has no leader
I20260923 17:31:09.821688 17146 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:09.822110 17146 ts_tablet_manager.cc:1397] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.000s
I20260923 17:31:09.823824 17164 tablet_replica.cc:330] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:09.824641 17164 raft_consensus.cc:2229] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.825081 17164 raft_consensus.cc:2258] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.825426 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 21fc7428f55041b6b939a8ca08574b90 (DEFAULT_TABLE table=default.Table_c64deefc885140dd972d0106708b6281 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.826583 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fc7428f55041b6b939a8ca08574b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.826622 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 21cf533908d04a3a8ca087355528556f (DEFAULT_TABLE table=default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.826897 17164 ts_tablet_manager.cc:1905] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.828678 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 21cf533908d04a3a8ca087355528556f (DEFAULT_TABLE table=default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.829737 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cf533908d04a3a8ca087355528556f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.831786 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 21fc7428f55041b6b939a8ca08574b90 (DEFAULT_TABLE table=default.Table_c64deefc885140dd972d0106708b6281 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.832026 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 21cf533908d04a3a8ca087355528556f (DEFAULT_TABLE table=default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.832983 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fc7428f55041b6b939a8ca08574b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.834154 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cf533908d04a3a8ca087355528556f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.836263 17146 raft_consensus.cc:348] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:09.837038 17146 raft_consensus.cc:374] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.837276 17146 raft_consensus.cc:729] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.837486 17165 raft_consensus.cc:2258] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.837872 17146 consensus_queue.cc:260] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:09.838434 16872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ca6aa0faa244799949fd44c5db72ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.839501 17165 ts_tablet_manager.cc:1905] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.840919 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fc7428f55041b6b939a8ca08574b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.840297 17017 tablet_service.cc:1467] Processing CreateTablet for tablet a4ca6aa0faa244799949fd44c5db72ed (DEFAULT_TABLE table=default.Table_7ef3741399e44b2fa037d322957f7deb [id=d0fff78552fb4121923c53b1a33d9994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.841729 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ca6aa0faa244799949fd44c5db72ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.844800 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cf533908d04a3a8ca087355528556f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.849177 16724 tablet_service.cc:1467] Processing CreateTablet for tablet a4ca6aa0faa244799949fd44c5db72ed (DEFAULT_TABLE table=default.Table_7ef3741399e44b2fa037d322957f7deb [id=d0fff78552fb4121923c53b1a33d9994]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.850409 16724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ca6aa0faa244799949fd44c5db72ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.875238 17146 ts_tablet_manager.cc:1428] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260923 17:31:09.879077 17146 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:09.829125 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 6a340a854bc54ac189a833e8de1bb8cf (table default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]) successfully deleted
W20260923 17:31:09.883405 16373 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 54299us
I20260923 17:31:09.885311 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ab69ba4df3cb4ec6a100b9c3e1c47205"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:09.886897 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab69ba4df3cb4ec6a100b9c3e1c47205 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:09.889357 16383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c9193016785148b492ccdc68f136e29b" } modify_external_catalogs: true
I20260923 17:31:09.868363 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 6a340a854bc54ac189a833e8de1bb8cf (table default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]) successfully deleted
I20260923 17:31:09.889837 17144 tablet_bootstrap.cc:654] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.890000 16383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c9193016785148b492ccdc68f136e29b" } modify_external_catalogs: true
W20260923 17:31:09.890055 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 66030us
I20260923 17:31:09.906032 17143 tablet_bootstrap.cc:492] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:09.906489 17143 ts_tablet_manager.cc:1397] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.142s user 0.010s sys 0.001s
I20260923 17:31:09.909438 17143 raft_consensus.cc:348] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.910260 17143 raft_consensus.cc:374] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:09.910511 17143 raft_consensus.cc:729] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:09.911140 17143 consensus_queue.cc:260] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.918453 17164 ts_tablet_manager.cc:1918] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.918825 17164 log.cc:1196] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/bf43276ac2e54b508722d899ce8cfe6e
I20260923 17:31:09.919476 17164 ts_tablet_manager.cc:1939] T bf43276ac2e54b508722d899ce8cfe6e P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:09.922981 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet bf43276ac2e54b508722d899ce8cfe6e (table default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]) successfully deleted
I20260923 17:31:09.944859 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c64deefc885140dd972d0106708b6281" } new_table_name: "default.Table_93cd61e86e5041d4adb389dc466c1062" modify_external_catalogs: true
I20260923 17:31:09.948590 17196 raft_consensus.cc:482] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:09.949148 17196 raft_consensus.cc:504] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:09.950802 17164 tablet_replica.cc:330] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:09.953963 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8154a3aed44924b7127bd4c8452c19" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:09.954959 16745 raft_consensus.cc:2454] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:09.957526 17164 raft_consensus.cc:2229] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:09.958137 17164 raft_consensus.cc:2258] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:09.961892 16380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3cc0a05ecc974d7cbd185095f39fee2b" } modify_external_catalogs: true
I20260923 17:31:09.962561 16380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3cc0a05ecc974d7cbd185095f39fee2b" } modify_external_catalogs: true
I20260923 17:31:09.963480 17164 ts_tablet_manager.cc:1905] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:09.964988 16826 leader_election.cc:302] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:09.966012 17147 raft_consensus.cc:2781] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:09.966300 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260923 17:31:09.966480 17147 raft_consensus.cc:482] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:09.966976 17147 raft_consensus.cc:3037] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:09.968818 17196 leader_election.cc:288] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:09.971084 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8154a3aed44924b7127bd4c8452c19" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:09.971871 17039 raft_consensus.cc:2454] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:09.973435 17165 ts_tablet_manager.cc:1918] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:09.973881 17165 log.cc:1196] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/6b38675078f8404db169774e32520b71
I20260923 17:31:09.974627 17165 ts_tablet_manager.cc:1939] T 6b38675078f8404db169774e32520b71 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:09.975111 17146 tablet_bootstrap.cc:654] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:09.981374 17143 ts_tablet_manager.cc:1428] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.075s user 0.006s sys 0.001s
I20260923 17:31:09.984258 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_6a55fbb756f44eb0800509b3e21e2134 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 1 got 0
I20260923 17:31:09.991655 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 29d44ee57faf4a1d99e077fd781a519c (DEFAULT_TABLE table=default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.992895 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d44ee57faf4a1d99e077fd781a519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.992832 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 6b38675078f8404db169774e32520b71 (table default.Table_3f641ab8052a4e93a7feec1a9d3d9562 [id=74563e8c9eaa480f86c62f576d025643]) successfully deleted
I20260923 17:31:09.993214 17017 tablet_service.cc:1467] Processing CreateTablet for tablet f124169d87c240a5bd544d3b6693323f (DEFAULT_TABLE table=default.Table_e9b25f5740c5469ab07a12b480144735 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.994483 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f124169d87c240a5bd544d3b6693323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.002194 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 29d44ee57faf4a1d99e077fd781a519c (DEFAULT_TABLE table=default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.003285 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d44ee57faf4a1d99e077fd781a519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:09.996326 17144 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:10.008409 17144 ts_tablet_manager.cc:1397] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.200s user 0.011s sys 0.009s
I20260923 17:31:10.009411 16723 tablet_service.cc:1467] Processing CreateTablet for tablet f124169d87c240a5bd544d3b6693323f (DEFAULT_TABLE table=default.Table_e9b25f5740c5469ab07a12b480144735 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.010807 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f124169d87c240a5bd544d3b6693323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.017361 16721 tablet_service.cc:1467] Processing CreateTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b (DEFAULT_TABLE table=default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.018864 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d22f0f9d7e6a46a08ca9f411cc2c658b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.021389 17143 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:10.021543 17146 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:10.021214 16871 tablet_service.cc:1467] Processing CreateTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b (DEFAULT_TABLE table=default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.021968 17146 ts_tablet_manager.cc:1397] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.143s user 0.017s sys 0.000s
I20260923 17:31:10.022506 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d22f0f9d7e6a46a08ca9f411cc2c658b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.023221 17021 tablet_service.cc:1467] Processing CreateTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b (DEFAULT_TABLE table=default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.024533 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d22f0f9d7e6a46a08ca9f411cc2c658b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.024592 17146 raft_consensus.cc:348] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.025288 17146 raft_consensus.cc:374] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.025528 17146 raft_consensus.cc:729] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.027601 17146 consensus_queue.cc:260] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.007772 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 29d44ee57faf4a1d99e077fd781a519c (DEFAULT_TABLE table=default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.029211 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d44ee57faf4a1d99e077fd781a519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.033582 17163 ts_tablet_manager.cc:1918] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.038316 17146 ts_tablet_manager.cc:1428] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260923 17:31:10.040324 17163 log.cc:1196] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/6a340a854bc54ac189a833e8de1bb8cf
I20260923 17:31:10.043383 16872 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5c465348d4e68a5047d764bb802b5 (DEFAULT_TABLE table=default.Table_ab80ba4ac749415083b0ebac2b5401e9 [id=98651426b57f456a9a13bd9bc11a27f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.044945 16872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5c465348d4e68a5047d764bb802b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.013727 16870 tablet_service.cc:1467] Processing CreateTablet for tablet f124169d87c240a5bd544d3b6693323f (DEFAULT_TABLE table=default.Table_e9b25f5740c5469ab07a12b480144735 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:09.987661 17147 raft_consensus.cc:504] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.054461 17146 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.055048 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8154a3aed44924b7127bd4c8452c19" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:10.055756 17039 raft_consensus.cc:3037] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.052953 17017 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5c465348d4e68a5047d764bb802b5 (DEFAULT_TABLE table=default.Table_ab80ba4ac749415083b0ebac2b5401e9 [id=98651426b57f456a9a13bd9bc11a27f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.056461 17164 ts_tablet_manager.cc:1918] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.058096 17164 log.cc:1196] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/45e1aa0b01b84ed091ff35792f7d4449
I20260923 17:31:10.058871 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f (DEFAULT_TABLE table=default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.059242 17164 ts_tablet_manager.cc:1939] T 45e1aa0b01b84ed091ff35792f7d4449 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:10.060801 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f (DEFAULT_TABLE table=default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.063710 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5c465348d4e68a5047d764bb802b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.064522 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a629ce6ab14d12b6a83ce6bc56f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.069932 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 8879dee684214c6dac3f9d6642a6d97d (DEFAULT_TABLE table=default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.071293 17039 raft_consensus.cc:2454] T 8e8154a3aed44924b7127bd4c8452c19 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:10.071623 17146 tablet_bootstrap.cc:654] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.072615 16827 leader_election.cc:302] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:10.073909 17196 raft_consensus.cc:2781] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:10.074932 17164 tablet_replica.cc:330] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:10.075294 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 45e1aa0b01b84ed091ff35792f7d4449 (table default.Table_745ed5dc2e06444ca3a30c0630398962 [id=0a7eb5754a6941318440a57c563ac1cd]) successfully deleted
I20260923 17:31:10.075717 17164 raft_consensus.cc:2229] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:10.076674 17164 pending_rounds.cc:62] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:10.076915 17164 pending_rounds.cc:68] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596406301794304 op_type: NO_OP noop_request { }
I20260923 17:31:10.077310 17164 raft_consensus.cc:2866] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:10.077632 17164 raft_consensus.cc:2258] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.078435 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ab80ba4ac749415083b0ebac2b5401e9" } new_table_name: "default.Table_cd593282564046359e7c7af93bc63eec" modify_external_catalogs: true
I20260923 17:31:10.085119 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f124169d87c240a5bd544d3b6693323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.086637 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3542e5d4b3e34da19c70ab50d8a95080"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:10.014935 17165 tablet_replica.cc:330] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:10.089260 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21fc7428f55041b6b939a8ca08574b90 consensus state has no leader
I20260923 17:31:10.089741 17165 raft_consensus.cc:2229] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:10.090489 17165 raft_consensus.cc:2258] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.095455 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a629ce6ab14d12b6a83ce6bc56f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.106832 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8879dee684214c6dac3f9d6642a6d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.051440 17147 leader_election.cc:288] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:10.109704 17147 raft_consensus.cc:482] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.110358 17147 raft_consensus.cc:504] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.111758 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 21a629ce6ab14d12b6a83ce6bc56f70f
I20260923 17:31:10.117075 16867 tablet_service.cc:1467] Processing CreateTablet for tablet a97e944f4e0c48db813e3bee61102c53 (DEFAULT_TABLE table=default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.121948 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_2eae26c78f434f09a58d8adc5cfc8bc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:10.123127 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 8879dee684214c6dac3f9d6642a6d97d (DEFAULT_TABLE table=default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.125854 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:10.129434 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2eae26c78f434f09a58d8adc5cfc8bc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:10.135130 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3542e5d4b3e34da19c70ab50d8a95080 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.139869 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:10.137959 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_28db4e4c622f47c9838b6d44c68d5499"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:10.161129 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28db4e4c622f47c9838b6d44c68d5499 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.146433 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_6a55fbb756f44eb0800509b3e21e2134 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 1 got 0
I20260923 17:31:10.168660 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_22a78bd0393140709e8a521e4480abf8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:10.172494 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22a78bd0393140709e8a521e4480abf8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.173667 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 1483835b2bc64873a3570ad17a079713 (DEFAULT_TABLE table=default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.183344 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97e944f4e0c48db813e3bee61102c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.037896 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5c465348d4e68a5047d764bb802b5 (DEFAULT_TABLE table=default.Table_ab80ba4ac749415083b0ebac2b5401e9 [id=98651426b57f456a9a13bd9bc11a27f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.186301 17163 ts_tablet_manager.cc:1939] T 6a340a854bc54ac189a833e8de1bb8cf P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:10.186975 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5c465348d4e68a5047d764bb802b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.197613 17021 tablet_service.cc:1467] Processing CreateTablet for tablet a97e944f4e0c48db813e3bee61102c53 (DEFAULT_TABLE table=default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.130379 17164 ts_tablet_manager.cc:1905] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.213331 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 1483835b2bc64873a3570ad17a079713 (DEFAULT_TABLE table=default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.217563 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8879dee684214c6dac3f9d6642a6d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.221374 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97e944f4e0c48db813e3bee61102c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.229513 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1483835b2bc64873a3570ad17a079713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.234259 17191 raft_consensus.cc:482] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.234997 17191 raft_consensus.cc:504] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.239497 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:10.240459 17191 leader_election.cc:288] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:10.240777 16746 raft_consensus.cc:2379] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5c071cdf65042a59cad226a15a5532c in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:10.248021 16380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc
I20260923 17:31:10.250631 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1483835b2bc64873a3570ad17a079713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.251232 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:10.252116 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 30d5c465348d4e68a5047d764bb802b5 consensus state has no leader
I20260923 17:31:10.257671 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:10.263430 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86f4b7ee14948f0989af54b3b77dc71" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:10.265798 16972 leader_election.cc:302] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:10.266911 17191 raft_consensus.cc:3037] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.274849 17191 raft_consensus.cc:2726] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:10.274775 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:10.035017 17144 raft_consensus.cc:348] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.053462 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8154a3aed44924b7127bd4c8452c19" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:10.280748 17144 raft_consensus.cc:374] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.280898 16745 raft_consensus.cc:3037] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.281092 17144 raft_consensus.cc:729] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.281757 17144 consensus_queue.cc:260] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.057107 16719 tablet_service.cc:1467] Processing CreateTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f (DEFAULT_TABLE table=default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.285595 16719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a629ce6ab14d12b6a83ce6bc56f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.067507 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 8879dee684214c6dac3f9d6642a6d97d (DEFAULT_TABLE table=default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.291096 17146 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:10.291235 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8879dee684214c6dac3f9d6642a6d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.292728 17146 ts_tablet_manager.cc:1397] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.241s user 0.023s sys 0.034s
I20260923 17:31:10.142759 16717 tablet_service.cc:1467] Processing CreateTablet for tablet a97e944f4e0c48db813e3bee61102c53 (DEFAULT_TABLE table=default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.293833 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97e944f4e0c48db813e3bee61102c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.154067 16715 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:10.151685 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 1483835b2bc64873a3570ad17a079713 (DEFAULT_TABLE table=default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.297237 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1483835b2bc64873a3570ad17a079713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.308964 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]). Expected version 1 got 0
I20260923 17:31:10.312009 17146 raft_consensus.cc:348] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.313958 17146 raft_consensus.cc:374] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.317674 17146 raft_consensus.cc:729] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.324131 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_203b4089c5ed4f77ba3eb3f0230ac163" } new_table_name: "default.Table_2ac2dde5fd1045f49876d98fabeda9de" modify_external_catalogs: true
I20260923 17:31:10.325070 17146 consensus_queue.cc:260] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.328863 16380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_793e622592644e9daae1b9dba1524dba" } modify_external_catalogs: true
I20260923 17:31:10.329680 16380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_793e622592644e9daae1b9dba1524dba" } modify_external_catalogs: true
I20260923 17:31:10.337608 17165 ts_tablet_manager.cc:1905] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.343770 17146 ts_tablet_manager.cc:1428] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260923 17:31:10.353951 17146 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.377066 17147 leader_election.cc:288] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:10.378592 17146 tablet_bootstrap.cc:654] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.380815 17143 tablet_bootstrap.cc:654] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.380935 17191 raft_consensus.cc:482] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.381568 17191 raft_consensus.cc:504] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.385275 17191 leader_election.cc:288] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.385926 17191 raft_consensus.cc:482] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.386508 17191 raft_consensus.cc:504] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.386947 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:10.389241 16972 leader_election.cc:341] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.394893 17191 leader_election.cc:288] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:10.398409 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:10.399523 16892 raft_consensus.cc:2454] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:10.401713 16972 leader_election.cc:302] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:10.405058 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:10.407063 17039 raft_consensus.cc:2454] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:10.407960 17164 ts_tablet_manager.cc:1918] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:10.408619 17164 log.cc:1196] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/6b38675078f8404db169774e32520b71
I20260923 17:31:10.412144 16827 leader_election.cc:302] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:10.418803 17147 raft_consensus.cc:2781] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:10.410074 17164 ts_tablet_manager.cc:1939] T 6b38675078f8404db169774e32520b71 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:10.419456 17147 raft_consensus.cc:482] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:10.420153 17147 raft_consensus.cc:3037] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.422040 17146 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:10.422425 17146 ts_tablet_manager.cc:1397] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.069s user 0.010s sys 0.005s
I20260923 17:31:10.425710 17146 raft_consensus.cc:348] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.426376 17146 raft_consensus.cc:374] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.426379 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 6b38675078f8404db169774e32520b71 (table default.Table_3f641ab8052a4e93a7feec1a9d3d9562 [id=74563e8c9eaa480f86c62f576d025643]) successfully deleted
I20260923 17:31:10.426622 17146 raft_consensus.cc:729] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.427160 17146 consensus_queue.cc:260] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.428359 17164 tablet_replica.cc:330] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:10.432698 17146 ts_tablet_manager.cc:1428] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260923 17:31:10.433499 17146 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.436506 17191 raft_consensus.cc:2781] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:10.437054 17191 raft_consensus.cc:482] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:10.437530 17191 raft_consensus.cc:3037] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.439642 17146 tablet_bootstrap.cc:654] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.440555 17164 raft_consensus.cc:2229] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:10.441169 17164 raft_consensus.cc:2258] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.442406 17163 tablet_replica.cc:330] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:10.443231 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:10.443966 16746 raft_consensus.cc:2454] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:10.446749 17163 raft_consensus.cc:2229] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:10.447342 17163 raft_consensus.cc:2258] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.449028 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:10.450667 16972 leader_election.cc:341] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.451746 16972 leader_election.cc:302] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:10.453688 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 6a340a854bc54ac189a833e8de1bb8cf (table default.Table_00de54e4eb1a42e388cdb06cdc068f18 [id=e1be42f7d37d4152ad3383a71435056d]) successfully deleted
I20260923 17:31:10.459040 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_d7bb20f8a5544384ab1e69311e05400e" } modify_external_catalogs: true
I20260923 17:31:10.459728 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_d7bb20f8a5544384ab1e69311e05400e" } modify_external_catalogs: true
I20260923 17:31:10.359249 17196 raft_consensus.cc:686] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:10.463254 17164 ts_tablet_manager.cc:1905] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.463033 16386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_e9b25f5740c5469ab07a12b480144735" } new_table_name: "default.Table_48c7725ec6034f21b65906e2c8c0f02c" modify_external_catalogs: true
I20260923 17:31:10.461113 17196 consensus_queue.cc:237] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.480196 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:10.480978 16746 raft_consensus.cc:2454] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:10.482357 17146 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
W20260923 17:31:10.482358 16725 consensus_meta.cc:287] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.555s user 0.010s sys 0.002s
I20260923 17:31:10.482867 17146 ts_tablet_manager.cc:1397] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.011s
I20260923 17:31:10.483063 17163 ts_tablet_manager.cc:1905] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.485617 17146 raft_consensus.cc:348] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:10.485939 16726 consensus_meta.cc:287] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.504s user 0.004s sys 0.006s
I20260923 17:31:10.486405 17146 raft_consensus.cc:374] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.486706 17146 raft_consensus.cc:729] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.487371 17146 consensus_queue.cc:260] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.489959 17218 raft_consensus.cc:2726] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:10.498716 17191 raft_consensus.cc:504] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.502327 17191 leader_election.cc:288] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:10.505215 16746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:10.505872 16746 raft_consensus.cc:3037] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.506481 16745 raft_consensus.cc:2454] T 8e8154a3aed44924b7127bd4c8452c19 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:10.532022 17146 ts_tablet_manager.cc:1428] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.049s user 0.011s sys 0.010s
I20260923 17:31:10.532935 17146 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.539173 17146 tablet_bootstrap.cc:654] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.531951 17164 ts_tablet_manager.cc:1918] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.556565 17164 log.cc:1196] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:10.557283 17164 ts_tablet_manager.cc:1939] T f3d473eff8734b6286c5162df107c3e6 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:10.557385 17146 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:10.557735 17146 ts_tablet_manager.cc:1397] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.000s
I20260923 17:31:10.559860 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet f3d473eff8734b6286c5162df107c3e6 (table default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]) successfully deleted
I20260923 17:31:10.560086 17146 raft_consensus.cc:348] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:10.560734 17146 raft_consensus.cc:374] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.561950 17146 raft_consensus.cc:729] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:10.563019 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 21a629ce6ab14d12b6a83ce6bc56f70f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21a629ce6ab14d12b6a83ce6bc56f70f already in progress: creating tablet
I20260923 17:31:10.562752 17146 consensus_queue.cc:260] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:10.570833 17164 tablet_replica.cc:330] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:10.571777 17146 ts_tablet_manager.cc:1428] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260923 17:31:10.572613 17146 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.575467 17164 raft_consensus.cc:2229] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:10.575950 17164 raft_consensus.cc:2258] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.577476 17164 ts_tablet_manager.cc:1905] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.578549 17146 tablet_bootstrap.cc:654] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:10.579375 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e2fa2e2d002746c8ba2b0dcf45372aa8 has no consensus state
I20260923 17:31:10.588634 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:10.602777 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_6a55fbb756f44eb0800509b3e21e2134 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 1 got 0
I20260923 17:31:10.603602 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 21fc7428f55041b6b939a8ca08574b90 (table default.Table_93cd61e86e5041d4adb389dc466c1062 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 1 got 0
I20260923 17:31:10.614905 17014 tablet_service.cc:1467] Processing CreateTablet for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (DEFAULT_TABLE table=default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.615971 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fa2e2d002746c8ba2b0dcf45372aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.615906 16726 tablet_service.cc:1467] Processing CreateTablet for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (DEFAULT_TABLE table=default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.618965 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 87b3350e2ed247e5b600405800993ddd (DEFAULT_TABLE table=default.Table_ab69ba4df3cb4ec6a100b9c3e1c47205 [id=01a5413d7f0a4b1fb5d69f17b0107814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.621197 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 87b3350e2ed247e5b600405800993ddd (DEFAULT_TABLE table=default.Table_ab69ba4df3cb4ec6a100b9c3e1c47205 [id=01a5413d7f0a4b1fb5d69f17b0107814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.622296 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b3350e2ed247e5b600405800993ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.625542 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 21fc7428f55041b6b939a8ca08574b90 (table default.Table_93cd61e86e5041d4adb389dc466c1062 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 1 got 0
I20260923 17:31:10.626228 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 21cf533908d04a3a8ca087355528556f (table default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]). Expected version 1 got 0
I20260923 17:31:10.626789 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a4ca6aa0faa244799949fd44c5db72ed (table default.Table_2c9fdbd8c49e4d9dbe68c5157000786d [id=d0fff78552fb4121923c53b1a33d9994]). Expected version 2 got 0
I20260923 17:31:10.549744 17144 ts_tablet_manager.cc:1428] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.541s user 0.017s sys 0.006s
W20260923 17:31:10.634639 17144 ts_tablet_manager.cc:1460] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Tablet startup took 625ms
I20260923 17:31:10.642813 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 87b3350e2ed247e5b600405800993ddd (DEFAULT_TABLE table=default.Table_ab69ba4df3cb4ec6a100b9c3e1c47205 [id=01a5413d7f0a4b1fb5d69f17b0107814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.645463 17164 ts_tablet_manager.cc:1918] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.645800 17164 log.cc:1196] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/3f0e90e28d114770a7c9f0e8d4d7fcfc
I20260923 17:31:10.645782 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b3350e2ed247e5b600405800993ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.646338 17164 ts_tablet_manager.cc:1939] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:10.651636 17213 consensus_queue.cc:1035] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:10.635003 17144 ts_tablet_manager.cc:1462] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Trace:
0923 17:31:09.732071 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:09.732413 (+ 342us) service_pool.cc:227] Handling call
0923 17:31:09.733880 (+ 1467us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:09.733957 (+ 77us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:09.750319 (+ 16362us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:09.783334 (+ 33015us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:09.808434 (+ 25100us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0923 17:31:09.809441 (+ 1007us) inbound_call.cc:173] Queueing success response
0923 17:31:10.008731 (+199290us) ts_tablet_manager.cc:1429] Starting tablet replica
0923 17:31:10.009763 (+ 1032us) tablet_replica.cc:259] Starting instrumentation
0923 17:31:10.010470 (+ 707us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":178417,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":2835,"tablet-open.queue_time_us":24727,"thread_start_us":265565,"threads_started":1}
I20260923 17:31:10.659795 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fa2e2d002746c8ba2b0dcf45372aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.660516 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b3350e2ed247e5b600405800993ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.675802 17222 raft_consensus.cc:482] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.676358 17222 raft_consensus.cc:504] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.679167 17222 leader_election.cc:288] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.681217 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:10.682082 16744 raft_consensus.cc:2454] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:10.682950 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:10.684587 16972 leader_election.cc:302] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:10.685472 17222 raft_consensus.cc:2781] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:10.685642 16972 leader_election.cc:341] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.685902 17222 raft_consensus.cc:482] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:10.686311 17222 raft_consensus.cc:3037] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:10.695372 17222 raft_consensus.cc:504] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.704566 17222 leader_election.cc:288] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.704644 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:10.712530 16972 leader_election.cc:341] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.716566 17144 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:10.718413 16871 tablet_service.cc:1467] Processing CreateTablet for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (DEFAULT_TABLE table=default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:10.719890 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fa2e2d002746c8ba2b0dcf45372aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:10.702560 17213 raft_consensus.cc:482] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.722265 17213 raft_consensus.cc:504] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
I20260923 17:31:10.724979 17213 leader_election.cc:288] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:10.730222 16827 leader_election.cc:341] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:10.730568 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (table default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]) successfully deleted
I20260923 17:31:10.731184 16379 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ad3626a8c0e24d85a90f0629af2c5b75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:10.733070 16379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad3626a8c0e24d85a90f0629af2c5b75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.737298 16746 raft_consensus.cc:2454] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:10.750247 17213 consensus_queue.cc:1035] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:10.750988 17222 raft_consensus.cc:482] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.751531 17222 raft_consensus.cc:504] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.753592 17222 leader_election.cc:288] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.758477 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:10.762497 16972 leader_election.cc:341] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.769990 17147 raft_consensus.cc:504] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } }
W20260923 17:31:10.771203 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63808fe254ec4b1f818ed3198d5aea3d consensus state has no leader
I20260923 17:31:10.783131 17144 tablet_bootstrap.cc:654] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.783603 16385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_93cd61e86e5041d4adb389dc466c1062" } new_table_name: "default.Table_f8418b500b374960b37e876bfc1b7a65" modify_external_catalogs: true
I20260923 17:31:10.787158 17163 ts_tablet_manager.cc:1918] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.787619 17163 log.cc:1196] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/6b38675078f8404db169774e32520b71
I20260923 17:31:10.787670 17222 raft_consensus.cc:482] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.788461 17163 ts_tablet_manager.cc:1939] T 6b38675078f8404db169774e32520b71 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:10.788177 17222 raft_consensus.cc:504] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.790294 17222 leader_election.cc:288] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.790745 17163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 21) took 1034 ms. Trace:
I20260923 17:31:10.797113 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_8bcef85f57544e91b397665c35757e48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:10.798642 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8bcef85f57544e91b397665c35757e48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:10.800189 16386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f124169d87c240a5bd544d3b6693323f consensus state has no leader
I20260923 17:31:10.809091 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_f03a6d12e8cb4f5298b3da28260ac01f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:10.810806 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f03a6d12e8cb4f5298b3da28260ac01f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.814059 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c6ffc66fd1a34a139ff3848a8e5effc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:10.828509 16388 catalog_manager.cc:5495] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:10.838280 16892 raft_consensus.cc:2379] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e5c071cdf65042a59cad226a15a5532c in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:10.789903 17143 tablet_bootstrap.cc:492] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:10.791335 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fc7428f55041b6b939a8ca08574b90" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:10.816298 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:10.844871 17143 ts_tablet_manager.cc:1397] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.824s user 0.014s sys 0.016s
W20260923 17:31:10.848634 16972 leader_election.cc:341] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:10.850169 17165 ts_tablet_manager.cc:1918] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260923 17:31:10.854904 16827 leader_election.cc:341] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 3f0e90e28d114770a7c9f0e8d4d7fcfc
I20260923 17:31:10.865165 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 8879dee684214c6dac3f9d6642a6d97d
I20260923 17:31:10.865290 16380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:10.872401 17165 log.cc:1196] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/bf43276ac2e54b508722d899ce8cfe6e
I20260923 17:31:10.873234 17165 ts_tablet_manager.cc:1939] T bf43276ac2e54b508722d899ce8cfe6e P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:10.873139 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:10.873955 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:10.876456 17147 leader_election.cc:288] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:10.888113 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:46075
--webserver_interface=127.15.246.188
--webserver_port=33399
--builtin_ntp_servers=127.15.246.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:31:10.892156 17146 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:10.893041 17146 ts_tablet_manager.cc:1397] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.321s user 0.090s sys 0.072s
W20260923 17:31:10.892980 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8879dee684214c6dac3f9d6642a6d97d already in progress: creating tablet
W20260923 17:31:10.893797 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 29d44ee57faf4a1d99e077fd781a519c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29d44ee57faf4a1d99e077fd781a519c already in progress: creating tablet
I20260923 17:31:10.847749 17143 raft_consensus.cc:348] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.900528 17143 raft_consensus.cc:374] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.900830 17143 raft_consensus.cc:729] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.905977 17165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 18) took 1130 ms. Trace:
I20260923 17:31:10.906352 17165 rpcz_store.cc:276] 0923 17:31:09.775617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:09.775780 (+ 163us) service_pool.cc:227] Handling call
0923 17:31:09.990508 (+214728us) ts_tablet_manager.cc:1145] Deleting tablet bf43276ac2e54b508722d899ce8cfe6e
0923 17:31:09.990557 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:10.584211 (+593654us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:10.849901 (+265690us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:10.905890 (+ 55989us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":214137}
I20260923 17:31:10.907874 17165 tablet_replica.cc:330] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:10.909713 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:10.909031 17143 consensus_queue.cc:260] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.910828 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:10.911526 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet bf43276ac2e54b508722d899ce8cfe6e (table default.Table_3707e7badee14016abf918e82796b952 [id=b07a5d058dfd46a0a1719d097c768252]) successfully deleted
I20260923 17:31:10.914058 17165 raft_consensus.cc:2229] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:10.920526 17165 raft_consensus.cc:2258] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:10.912768 17146 raft_consensus.cc:348] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:10.922492 17146 raft_consensus.cc:374] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:10.922794 17146 raft_consensus.cc:729] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:10.924093 17146 consensus_queue.cc:260] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:10.916142 17143 ts_tablet_manager.cc:1428] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.071s user 0.005s sys 0.000s
W20260923 17:31:10.935102 16385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21fc7428f55041b6b939a8ca08574b90 consensus state has no leader
I20260923 17:31:10.937459 17165 ts_tablet_manager.cc:1905] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:10.939644 17143 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:10.948611 17146 ts_tablet_manager.cc:1428] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20260923 17:31:10.952674 17146 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:10.955881 17147 raft_consensus.cc:482] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:10.957043 17147 raft_consensus.cc:504] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:10.959769 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:10.960757 17039 raft_consensus.cc:2379] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:10.962235 17143 tablet_bootstrap.cc:654] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.973187 17147 leader_election.cc:288] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:10.985263 17146 tablet_bootstrap.cc:654] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:10.995700 17165 ts_tablet_manager.cc:1918] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:10.996129 17165 log.cc:1196] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:11.001633 17165 ts_tablet_manager.cc:1939] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:11.002418 17165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 20) took 1198 ms. Trace:
I20260923 17:31:11.002731 17165 rpcz_store.cc:276] 0923 17:31:09.804490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:09.804637 (+ 147us) service_pool.cc:227] Handling call
0923 17:31:10.907716 (+1103079us) ts_tablet_manager.cc:1145] Deleting tablet f3d473eff8734b6286c5162df107c3e6
0923 17:31:10.907756 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:10.987815 (+ 80059us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:10.995464 (+ 7649us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:11.002364 (+ 6900us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5922,"tablet-delete.queue_time_us":1102459}
I20260923 17:31:11.004273 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet f3d473eff8734b6286c5162df107c3e6 (table default.Table_392d5ef452e04a2e9013847c2f2e6948 [id=8ce6602d0eca41e893b5078f34a07265]) successfully deleted
I20260923 17:31:11.005326 17165 tablet_replica.cc:330] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:11.005447 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 21a629ce6ab14d12b6a83ce6bc56f70f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21a629ce6ab14d12b6a83ce6bc56f70f already in progress: creating tablet
I20260923 17:31:11.008910 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2eae26c78f434f09a58d8adc5cfc8bc5" } new_table_name: "default.Table_89ce6a7e0e1c412aa40457102938869c" modify_external_catalogs: true
I20260923 17:31:11.006086 17165 raft_consensus.cc:2229] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:11.013340 17165 raft_consensus.cc:2258] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:11.017107 17165 ts_tablet_manager.cc:1905] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:11.025656 17222 raft_consensus.cc:482] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.026234 17222 raft_consensus.cc:504] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.028507 17222 leader_election.cc:288] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:11.029527 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:11.031111 16972 leader_election.cc:341] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.046593 17165 ts_tablet_manager.cc:1918] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:11.046942 17165 log.cc:1196] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/3f0e90e28d114770a7c9f0e8d4d7fcfc
I20260923 17:31:11.047220 16379 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_6a55fbb756f44eb0800509b3e21e2134 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 1 got 0
I20260923 17:31:11.047513 17165 ts_tablet_manager.cc:1939] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:11.049154 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (table default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]) successfully deleted
I20260923 17:31:11.049831 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f (DEFAULT_TABLE table=default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.050930 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d65cfb2eb3648faab5869bba69cab3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.052424 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f (DEFAULT_TABLE table=default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.053486 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d65cfb2eb3648faab5869bba69cab3f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:11.056043 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 29d44ee57faf4a1d99e077fd781a519c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29d44ee57faf4a1d99e077fd781a519c already in progress: creating tablet
W20260923 17:31:11.059082 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8879dee684214c6dac3f9d6642a6d97d already in progress: creating tablet
I20260923 17:31:11.062677 17016 tablet_service.cc:1467] Processing CreateTablet for tablet cbb9840f8b6d464caae440fb03a7be54 (DEFAULT_TABLE table=default.Table_2eae26c78f434f09a58d8adc5cfc8bc5 [id=e8986668f81448488cbf75f8bf0e24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.063939 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb9840f8b6d464caae440fb03a7be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.066376 16867 tablet_service.cc:1467] Processing CreateTablet for tablet cbb9840f8b6d464caae440fb03a7be54 (DEFAULT_TABLE table=default.Table_2eae26c78f434f09a58d8adc5cfc8bc5 [id=e8986668f81448488cbf75f8bf0e24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.067207 16865 tablet_service.cc:1467] Processing CreateTablet for tablet ceb787c0863847289400e62718d218a1 (DEFAULT_TABLE table=default.Table_22a78bd0393140709e8a521e4480abf8 [id=f3dfb81ee57147d79f9c67dbbaf451b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.067713 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb9840f8b6d464caae440fb03a7be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.068343 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb787c0863847289400e62718d218a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.073109 17021 tablet_service.cc:1467] Processing CreateTablet for tablet ceb787c0863847289400e62718d218a1 (DEFAULT_TABLE table=default.Table_22a78bd0393140709e8a521e4480abf8 [id=f3dfb81ee57147d79f9c67dbbaf451b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.074236 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb787c0863847289400e62718d218a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.076992 16870 tablet_service.cc:1467] Processing CreateTablet for tablet df53a9825e5348189485f30b8fdedc63 (DEFAULT_TABLE table=default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.077953 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df53a9825e5348189485f30b8fdedc63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:11.086993 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cbb9840f8b6d464caae440fb03a7be54 consensus state has no leader
I20260923 17:31:11.088649 16386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_28db4e4c622f47c9838b6d44c68d5499" } modify_external_catalogs: true
I20260923 17:31:11.089234 16386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_28db4e4c622f47c9838b6d44c68d5499" } modify_external_catalogs: true
I20260923 17:31:11.094756 17018 tablet_service.cc:1467] Processing CreateTablet for tablet df53a9825e5348189485f30b8fdedc63 (DEFAULT_TABLE table=default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.095942 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df53a9825e5348189485f30b8fdedc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.104876 16379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b8377c11b1cc4acebc14e04381f6c57b" } modify_external_catalogs: true
I20260923 17:31:11.105543 16379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b8377c11b1cc4acebc14e04381f6c57b" } modify_external_catalogs: true
I20260923 17:31:11.148487 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b518db3a4e274e35a617e1c2b35f43e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:11.150403 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b518db3a4e274e35a617e1c2b35f43e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.157087 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ce78da2e733545078e9762632b134528"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:11.159157 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce78da2e733545078e9762632b134528 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.168814 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_8bcef85f57544e91b397665c35757e48" } new_table_name: "default.Table_1fcc5b4a9266403f911dafeac141ac3f" modify_external_catalogs: true
I20260923 17:31:11.177596 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c925434b48d5441eab0ddbc29bed59f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:11.179455 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c925434b48d5441eab0ddbc29bed59f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.186823 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:11.190156 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:11.191841 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:11.191885 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 8d65cfb2eb3648faab5869bba69cab3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d65cfb2eb3648faab5869bba69cab3f already in progress: creating tablet
W20260923 17:31:11.193580 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 8d65cfb2eb3648faab5869bba69cab3f because the tablet was not found. No further retry: Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:11.223363 16379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 1483835b2bc64873a3570ad17a079713
W20260923 17:31:11.229650 16716 consensus_meta.cc:287] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.717s user 0.004s sys 0.009s
I20260923 17:31:11.231693 16972 leader_election.cc:302] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
W20260923 17:31:11.232380 16972 raft_consensus.cc:2670] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P e5c071cdf65042a59cad226a15a5532c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:11.240797 17222 raft_consensus.cc:482] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.241308 17222 raft_consensus.cc:504] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.243273 17222 leader_election.cc:288] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:11.245355 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:11.247929 16972 leader_election.cc:341] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.232090 16716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42676 (request call id 33) took 1087 ms. Trace:
I20260923 17:31:11.262179 16716 rpcz_store.cc:276] 0923 17:31:10.144345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:10.147862 (+ 3517us) service_pool.cc:227] Handling call
0923 17:31:10.296740 (+148878us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:10.296824 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:10.511426 (+214602us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:11.230995 (+719569us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:11.231944 (+ 949us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:11.267572 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:11.260996 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_be66ae961c4342319e013d42905625d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:11.269701 16719 tablet_service.cc:1467] Processing CreateTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f (DEFAULT_TABLE table=default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:11.269750 16717 consensus_meta.cc:287] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.758s user 0.002s sys 0.008s
I20260923 17:31:11.271080 16719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d65cfb2eb3648faab5869bba69cab3f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:11.271188 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be66ae961c4342319e013d42905625d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.272145 16717 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42676 (request call id 32) took 1165 ms. Trace:
I20260923 17:31:11.272553 16717 rpcz_store.cc:276] 0923 17:31:10.107010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:10.107189 (+ 179us) service_pool.cc:227] Handling call
0923 17:31:10.293354 (+186165us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:10.293438 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:10.511647 (+218209us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:11.271543 (+759896us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:11.272082 (+ 539us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:11.273411 16723 tablet_service.cc:1467] Processing CreateTablet for tablet df53a9825e5348189485f30b8fdedc63 (DEFAULT_TABLE table=default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.274516 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df53a9825e5348189485f30b8fdedc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.276119 17143 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:11.277109 16725 tablet_service.cc:1467] Processing CreateTablet for tablet cbb9840f8b6d464caae440fb03a7be54 (DEFAULT_TABLE table=default.Table_2eae26c78f434f09a58d8adc5cfc8bc5 [id=e8986668f81448488cbf75f8bf0e24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.278160 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbb9840f8b6d464caae440fb03a7be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.278201 17143 ts_tablet_manager.cc:1397] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.347s user 0.039s sys 0.051s
I20260923 17:31:11.280807 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:11.281905 16738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.283550 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.284714 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fc7428f55041b6b939a8ca08574b90" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.280604 17143 raft_consensus.cc:348] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.286060 17143 raft_consensus.cc:374] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.285920 16741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.286332 17143 raft_consensus.cc:729] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.286962 17143 consensus_queue.cc:260] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.272570 16721 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:11.290956 16720 tablet_service.cc:1467] Processing CreateTablet for tablet ceb787c0863847289400e62718d218a1 (DEFAULT_TABLE table=default.Table_22a78bd0393140709e8a521e4480abf8 [id=f3dfb81ee57147d79f9c67dbbaf451b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.291414 17143 ts_tablet_manager.cc:1428] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260923 17:31:11.292002 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceb787c0863847289400e62718d218a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.292361 17143 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:11.295454 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:11.296558 16972 leader_election.cc:341] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.296964 16972 leader_election.cc:302] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.297729 17143 tablet_bootstrap.cc:654] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.297696 17222 raft_consensus.cc:2726] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:11.297964 16972 leader_election.cc:341] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.298349 16972 leader_election.cc:302] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.298991 17222 raft_consensus.cc:2726] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:11.299336 16972 leader_election.cc:341] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.300967 16721 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:11.302772 16826 leader_election.cc:341] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f3d473eff8734b6286c5162df107c3e6
I20260923 17:31:11.303194 16826 leader_election.cc:302] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:11.303732 16826 raft_consensus.cc:2670] T f3d473eff8734b6286c5162df107c3e6 P c36e79694e154f84a46980719217d7f2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:11.287307 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:11.304915 16742 raft_consensus.cc:3037] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.299718 16972 leader_election.cc:302] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:11.307370 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1483835b2bc64873a3570ad17a079713 already in progress: creating tablet
I20260923 17:31:11.312366 16387 master_service.cc:946] ReplaceTablet: received request to replace tablet 21fc7428f55041b6b939a8ca08574b90 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:11.312875 16380 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_6a55fbb756f44eb0800509b3e21e2134" } new_table_name: "default.Table_bdb104858f024c16a05173856e5c422e" modify_external_catalogs: true
I20260923 17:31:11.316545 17222 raft_consensus.cc:2726] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:11.325692 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0721fa178c0347598351b9d3512089b8" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.328295 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0e90e28d114770a7c9f0e8d4d7fcfc" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:11.329036 16744 raft_consensus.cc:2379] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:10.791137 17163 rpcz_store.cc:276] 0923 17:31:09.755999 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:09.756320 (+ 321us) service_pool.cc:227] Handling call
0923 17:31:10.442215 (+685895us) ts_tablet_manager.cc:1145] Deleting tablet 6b38675078f8404db169774e32520b71
0923 17:31:10.442268 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:10.676363 (+234095us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:10.786881 (+110518us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:10.790672 (+ 3791us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3323,"tablet-delete.queue_time_us":665451}
I20260923 17:31:11.330242 17163 tablet_replica.cc:330] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:11.334494 17163 raft_consensus.cc:2229] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:11.334990 17163 raft_consensus.cc:2258] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:11.340515 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 6b38675078f8404db169774e32520b71 (table default.Table_3f641ab8052a4e93a7feec1a9d3d9562 [id=74563e8c9eaa480f86c62f576d025643]) successfully deleted
W20260923 17:31:11.343286 17144 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.558s user 0.005s sys 0.000s
I20260923 17:31:11.362141 17146 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.362569 17146 ts_tablet_manager.cc:1397] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.411s user 0.051s sys 0.080s
I20260923 17:31:11.362599 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:11.363263 16826 leader_election.cc:302] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:11.364500 16972 leader_election.cc:341] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:11.364789 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1483835b2bc64873a3570ad17a079713 already in progress: creating tablet
I20260923 17:31:11.353142 17163 ts_tablet_manager.cc:1905] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:11.367004 17196 raft_consensus.cc:3037] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.374248 17196 raft_consensus.cc:2726] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:11.365036 17144 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.579s user 0.006s sys 0.001s
I20260923 17:31:11.377979 17144 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:11.364779 17146 raft_consensus.cc:348] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.378422 17146 raft_consensus.cc:374] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.378705 17146 raft_consensus.cc:729] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.379441 17146 consensus_queue.cc:260] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.378590 16972 leader_election.cc:302] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.381405 17146 ts_tablet_manager.cc:1428] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260923 17:31:11.384691 17222 raft_consensus.cc:2726] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:11.385079 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8895cffac5945a19a5de7fba1319f22 has no consensus state
I20260923 17:31:11.396989 17144 ts_tablet_manager.cc:1397] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.681s user 0.012s sys 0.004s
I20260923 17:31:11.375387 16826 leader_election.cc:302] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:11.400929 16826 raft_consensus.cc:2670] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P c36e79694e154f84a46980719217d7f2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:11.400701 16379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c6ffc66fd1a34a139ff3848a8e5effc6" } modify_external_catalogs: true
I20260923 17:31:11.401433 16379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c6ffc66fd1a34a139ff3848a8e5effc6" } modify_external_catalogs: true
I20260923 17:31:11.382222 17146 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.405865 17144 raft_consensus.cc:348] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.407622 17146 tablet_bootstrap.cc:654] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:11.420317 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 45003.5us
W20260923 17:31:11.353544 16352 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 89175.7us
I20260923 17:31:11.424494 17196 consensus_queue.cc:1035] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:11.437285 16384 master_service.cc:946] ReplaceTablet: received request to replace tablet 21fc7428f55041b6b939a8ca08574b90 from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:11.438676 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 74203.9us
I20260923 17:31:11.440960 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:11.442912 17164 tablet_replica.cc:330] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:11.443579 17164 raft_consensus.cc:2229] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:11.443961 17164 raft_consensus.cc:2258] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:11.445865 17164 ts_tablet_manager.cc:1905] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:11.449674 17249 raft_consensus.cc:482] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.450143 17249 raft_consensus.cc:504] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.452025 17249 leader_election.cc:288] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:11.456733 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:11.457415 16893 raft_consensus.cc:2454] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:11.399621 16742 raft_consensus.cc:2454] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:11.462025 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.478967 17164 ts_tablet_manager.cc:1918] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:11.490634 16385 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a4ca6aa0faa244799949fd44c5db72ed (table default.Table_2c9fdbd8c49e4d9dbe68c5157000786d [id=d0fff78552fb4121923c53b1a33d9994]). Expected version 2 got 0
I20260923 17:31:11.493427 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_6a55fbb756f44eb0800509b3e21e2134 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 1 got 0
I20260923 17:31:11.493568 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 723000dc175844e990740dc503b53ced (DEFAULT_TABLE table=default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.494626 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723000dc175844e990740dc503b53ced. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:11.494949 16380 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0721fa178c0347598351b9d3512089b8 consensus state has no leader
I20260923 17:31:11.507906 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9b734b250d4920abbafe76f52ec540 (DEFAULT_TABLE table=default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.509121 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9b734b250d4920abbafe76f52ec540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.508256 16871 tablet_service.cc:1467] Processing CreateTablet for tablet a3bcaddfad4f45739a705c17bdfe332f (DEFAULT_TABLE table=default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.510332 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bcaddfad4f45739a705c17bdfe332f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.509163 16865 tablet_service.cc:1467] Processing CreateTablet for tablet c8895cffac5945a19a5de7fba1319f22 (DEFAULT_TABLE table=default.Table_1fcc5b4a9266403f911dafeac141ac3f [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.513578 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8895cffac5945a19a5de7fba1319f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.514791 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_40b23e3d14dd4c1da5acb4c38e0e466f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:11.516814 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40b23e3d14dd4c1da5acb4c38e0e466f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.524860 17164 log.cc:1196] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:11.525107 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 723000dc175844e990740dc503b53ced (DEFAULT_TABLE table=default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.525488 17164 ts_tablet_manager.cc:1939] T 29d44ee57faf4a1d99e077fd781a519c P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:11.525821 17146 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.526180 17146 ts_tablet_manager.cc:1397] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.144s user 0.011s sys 0.013s
I20260923 17:31:11.526347 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723000dc175844e990740dc503b53ced. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:11.480626 16972 leader_election.cc:341] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.491858 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9b734b250d4920abbafe76f52ec540 (DEFAULT_TABLE table=default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:11.528426 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 47840.4us
I20260923 17:31:11.459604 17144 raft_consensus.cc:374] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.495788 17018 tablet_service.cc:1467] Processing CreateTablet for tablet a3bcaddfad4f45739a705c17bdfe332f (DEFAULT_TABLE table=default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.529035 16972 leader_election.cc:302] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.498649 17021 tablet_service.cc:1467] Processing CreateTablet for tablet c8895cffac5945a19a5de7fba1319f22 (DEFAULT_TABLE table=default.Table_1fcc5b4a9266403f911dafeac141ac3f [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.530119 16972 leader_election.cc:302] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217
I20260923 17:31:11.528999 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9b734b250d4920abbafe76f52ec540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.531118 17249 raft_consensus.cc:2781] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:11.528879 17144 raft_consensus.cc:729] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.531805 17222 raft_consensus.cc:2781] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:11.532117 17222 raft_consensus.cc:482] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:11.532459 17222 raft_consensus.cc:3037] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.535149 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 723000dc175844e990740dc503b53ced (DEFAULT_TABLE table=default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.536208 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723000dc175844e990740dc503b53ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.536511 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8895cffac5945a19a5de7fba1319f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.537806 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9b734b250d4920abbafe76f52ec540 (DEFAULT_TABLE table=default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.538892 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9b734b250d4920abbafe76f52ec540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.540532 16717 tablet_service.cc:1467] Processing CreateTablet for tablet a3bcaddfad4f45739a705c17bdfe332f (DEFAULT_TABLE table=default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.540849 17146 raft_consensus.cc:348] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.541517 17146 raft_consensus.cc:374] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.541553 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bcaddfad4f45739a705c17bdfe332f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.541728 17146 raft_consensus.cc:729] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.542176 17146 consensus_queue.cc:260] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.544111 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bcaddfad4f45739a705c17bdfe332f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.545166 17222 raft_consensus.cc:504] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.549522 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 29d44ee57faf4a1d99e077fd781a519c (table default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]) successfully deleted
I20260923 17:31:11.552062 16718 tablet_service.cc:1467] Processing CreateTablet for tablet c8895cffac5945a19a5de7fba1319f22 (DEFAULT_TABLE table=default.Table_1fcc5b4a9266403f911dafeac141ac3f [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:11.532296 17144 consensus_queue.cc:260] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.553287 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8895cffac5945a19a5de7fba1319f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:11.554461 17144 ts_tablet_manager.cc:1428] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.151s user 0.006s sys 0.000s
I20260923 17:31:11.558874 17146 ts_tablet_manager.cc:1428] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.032s user 0.005s sys 0.001s
I20260923 17:31:11.559727 17146 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.561187 17144 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:11.568648 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:11.569267 16893 raft_consensus.cc:3037] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.576771 16893 raft_consensus.cc:2454] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:11.577394 16385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2c9fdbd8c49e4d9dbe68c5157000786d" } modify_external_catalogs: true
I20260923 17:31:11.577900 16972 leader_election.cc:302] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:11.578092 16385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2c9fdbd8c49e4d9dbe68c5157000786d" } modify_external_catalogs: true
I20260923 17:31:11.585426 17222 leader_election.cc:288] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:11.585661 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:11.585911 17222 raft_consensus.cc:2781] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
W20260923 17:31:11.586766 16972 leader_election.cc:341] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.586270 17222 raft_consensus.cc:686] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:11.596624 17222 consensus_queue.cc:237] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.598098 17249 raft_consensus.cc:686] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:11.601758 17249 consensus_queue.cc:237] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.602530 17143 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:11.602981 17143 ts_tablet_manager.cc:1397] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.311s user 0.088s sys 0.047s
I20260923 17:31:11.545775 17163 ts_tablet_manager.cc:1918] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:11.609715 17163 log.cc:1196] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/3f0e90e28d114770a7c9f0e8d4d7fcfc
I20260923 17:31:11.610584 17163 ts_tablet_manager.cc:1939] T 3f0e90e28d114770a7c9f0e8d4d7fcfc P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:11.612349 17163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 35) took 1361 ms. Trace:
I20260923 17:31:11.601037 17146 tablet_bootstrap.cc:654] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.623589 17144 tablet_bootstrap.cc:654] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.636008 17143 raft_consensus.cc:348] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.636786 17143 raft_consensus.cc:374] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.637050 17143 raft_consensus.cc:729] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.637818 17143 consensus_queue.cc:260] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.640250 17146 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.640671 17146 ts_tablet_manager.cc:1397] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.081s user 0.009s sys 0.000s
I20260923 17:31:11.612620 17163 rpcz_store.cc:276] 0923 17:31:10.250747 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:10.251019 (+ 272us) service_pool.cc:227] Handling call
0923 17:31:11.330006 (+1078987us) ts_tablet_manager.cc:1145] Deleting tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc
0923 17:31:11.330061 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:11.479709 (+149648us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:11.520055 (+ 40346us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:11.612276 (+ 92221us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3472,"tablet-delete.queue_time_us":1078330}
I20260923 17:31:11.645385 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 3f0e90e28d114770a7c9f0e8d4d7fcfc (table default.Table_3cc0a05ecc974d7cbd185095f39fee2b [id=9d28901f52cd4b9cac3e87408a71c985]) successfully deleted
I20260923 17:31:11.646494 17163 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 34) took 1501 ms. Trace:
I20260923 17:31:11.646811 17163 rpcz_store.cc:276] 0923 17:31:10.145247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:10.148375 (+ 3128us) service_pool.cc:227] Handling call
0923 17:31:11.646133 (+1497758us) ts_tablet_manager.cc:1145] Deleting tablet 21a629ce6ab14d12b6a83ce6bc56f70f
0923 17:31:11.646180 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:11.646441 (+ 261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1349690}
W20260923 17:31:11.647877 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 21a629ce6ab14d12b6a83ce6bc56f70f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21a629ce6ab14d12b6a83ce6bc56f70f already in progress: creating tablet
I20260923 17:31:11.649106 17196 consensus_queue.cc:1035] T 8e8154a3aed44924b7127bd4c8452c19 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:11.649587 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8879dee684214c6dac3f9d6642a6d97d already in progress: creating tablet
W20260923 17:31:11.650820 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 29d44ee57faf4a1d99e077fd781a519c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29d44ee57faf4a1d99e077fd781a519c already in progress: creating tablet
W20260923 17:31:11.652166 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 8d65cfb2eb3648faab5869bba69cab3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d65cfb2eb3648faab5869bba69cab3f already in progress: creating tablet
I20260923 17:31:11.643364 17146 raft_consensus.cc:348] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.652518 17146 raft_consensus.cc:374] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.652750 17146 raft_consensus.cc:729] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.653373 17146 consensus_queue.cc:260] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.658363 17146 ts_tablet_manager.cc:1428] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260923 17:31:11.666024 17146 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:11.666193 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1483835b2bc64873a3570ad17a079713 already in progress: creating tablet
I20260923 17:31:11.670975 17146 tablet_bootstrap.cc:654] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.672608 17144 tablet_bootstrap.cc:492] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:11.672641 17143 ts_tablet_manager.cc:1428] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.069s user 0.008s sys 0.000s
I20260923 17:31:11.673098 17144 ts_tablet_manager.cc:1397] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.006s
I20260923 17:31:11.675359 17144 raft_consensus.cc:348] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.676142 17143 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:11.680002 17144 raft_consensus.cc:374] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.680370 17144 raft_consensus.cc:729] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.681075 17144 consensus_queue.cc:260] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.683544 17144 ts_tablet_manager.cc:1428] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.010s user 0.000s sys 0.004s
I20260923 17:31:11.684365 17144 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:11.697165 17147 raft_consensus.cc:482] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.697599 17147 raft_consensus.cc:504] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.699396 17147 leader_election.cc:288] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:11.702016 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:11.702661 17039 raft_consensus.cc:2454] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:11.704087 16827 leader_election.cc:302] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:11.705099 17147 raft_consensus.cc:2781] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:11.705502 17147 raft_consensus.cc:482] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:11.705873 17147 raft_consensus.cc:3037] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.702399 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:11.709018 16826 leader_election.cc:341] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.710023 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_1fcc5b4a9266403f911dafeac141ac3f" } new_table_name: "default.Table_0d464342027a447fbc67d549f98de051" modify_external_catalogs: true
I20260923 17:31:11.717717 17146 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.718112 17146 ts_tablet_manager.cc:1397] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.059s user 0.007s sys 0.003s
I20260923 17:31:11.718048 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]). Expected version 1 got 0
I20260923 17:31:11.720131 17146 raft_consensus.cc:348] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.720579 17143 tablet_bootstrap.cc:654] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.720798 17146 raft_consensus.cc:374] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.721032 17146 raft_consensus.cc:729] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.721570 17146 consensus_queue.cc:260] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.723357 17146 ts_tablet_manager.cc:1428] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:11.735743 17144 tablet_bootstrap.cc:654] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.742189 17146 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.754788 17144 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:11.755307 17144 ts_tablet_manager.cc:1397] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.006s
I20260923 17:31:11.755990 17146 tablet_bootstrap.cc:654] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.757870 17144 raft_consensus.cc:348] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.758621 17144 raft_consensus.cc:374] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.758855 17144 raft_consensus.cc:729] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.759460 17144 consensus_queue.cc:260] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.764612 17144 ts_tablet_manager.cc:1428] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260923 17:31:11.768256 17144 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:11.777501 17144 tablet_bootstrap.cc:654] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.777501 17146 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.777882 17146 ts_tablet_manager.cc:1397] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.004s
I20260923 17:31:11.779830 17146 raft_consensus.cc:348] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.780473 17146 raft_consensus.cc:374] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.780697 17146 raft_consensus.cc:729] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.781273 17146 consensus_queue.cc:260] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.783020 17146 ts_tablet_manager.cc:1428] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:11.788196 17146 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.789994 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:11.791446 16387 catalog_manager.cc:6565] ReplaceTablet: tablet 21fc7428f55041b6b939a8ca08574b90 deleted and replaced by tablet e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:11.794394 17143 tablet_bootstrap.cc:492] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:11.794996 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:11.795665 17143 ts_tablet_manager.cc:1397] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.120s user 0.008s sys 0.005s
I20260923 17:31:11.796820 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:11.796831 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:11.798242 17164 tablet_replica.cc:330] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
W20260923 17:31:11.798584 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 21fc7428f55041b6b939a8ca08574b90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21fc7428f55041b6b939a8ca08574b90 already in progress: creating tablet
I20260923 17:31:11.799083 17164 raft_consensus.cc:2229] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:11.799523 17164 raft_consensus.cc:2258] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:11.800298 16387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_cd593282564046359e7c7af93bc63eec" } new_table_name: "default.Table_af6ea13bdd26434f9222c9255b68ee1e" modify_external_catalogs: true
I20260923 17:31:11.801230 17164 ts_tablet_manager.cc:1905] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:11.801546 17144 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:11.801899 17144 ts_tablet_manager.cc:1397] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.005s
I20260923 17:31:11.802661 17146 tablet_bootstrap.cc:654] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:11.802595 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63808fe254ec4b1f818ed3198d5aea3d consensus state has no leader
I20260923 17:31:11.805219 17144 raft_consensus.cc:348] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.805799 17144 raft_consensus.cc:374] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.806013 17144 raft_consensus.cc:729] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.808494 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_2d35179a997541eda78f604ede819340"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:11.809235 17144 consensus_queue.cc:260] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.814136 17144 ts_tablet_manager.cc:1428] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20260923 17:31:11.819203 17144 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:11.820873 17252 raft_consensus.cc:482] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.821393 17252 raft_consensus.cc:504] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.823608 17252 leader_election.cc:288] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:11.826005 17143 raft_consensus.cc:348] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.826747 17143 raft_consensus.cc:374] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.826992 17143 raft_consensus.cc:729] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.826865 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d44ee57faf4a1d99e077fd781a519c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:11.828383 16677 leader_election.cc:341] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.834263 17146 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.834635 17146 ts_tablet_manager.cc:1397] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.012s
I20260923 17:31:11.836630 17146 raft_consensus.cc:348] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.837276 17146 raft_consensus.cc:374] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.837522 17146 raft_consensus.cc:729] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.838095 17146 consensus_queue.cc:260] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.841084 17146 ts_tablet_manager.cc:1428] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260923 17:31:11.841938 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d35179a997541eda78f604ede819340 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.841800 17143 consensus_queue.cc:260] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.844565 17146 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:11.847333 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 21fc7428f55041b6b939a8ca08574b90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21fc7428f55041b6b939a8ca08574b90 already in progress: creating tablet
I20260923 17:31:11.862252 17146 tablet_bootstrap.cc:654] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.866870 17143 ts_tablet_manager.cc:1428] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.071s user 0.006s sys 0.000s
W20260923 17:31:11.868806 16678 leader_election.cc:341] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:11.869280 16678 leader_election.cc:302] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:11.869385 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_46a8dbb79853475b8593a2de25a1bc3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:11.870546 17252 raft_consensus.cc:2726] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:11.871129 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46a8dbb79853475b8593a2de25a1bc3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.873687 17146 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.874060 17146 ts_tablet_manager.cc:1397] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.002s
I20260923 17:31:11.874557 17144 tablet_bootstrap.cc:654] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.876098 17146 raft_consensus.cc:348] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.876699 17146 raft_consensus.cc:374] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.876945 17146 raft_consensus.cc:729] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.877483 17146 consensus_queue.cc:260] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.879485 17143 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:11.879577 17146 ts_tablet_manager.cc:1428] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:11.880331 17146 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.880270 17252 raft_consensus.cc:482] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.880681 17252 raft_consensus.cc:504] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.884466 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:11.884562 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ca6aa0faa244799949fd44c5db72ed" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:11.884873 17143 tablet_bootstrap.cc:654] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.885221 16893 raft_consensus.cc:2379] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:11.882395 17252 leader_election.cc:288] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:11.886253 16677 leader_election.cc:302] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:11.887022 17252 raft_consensus.cc:3037] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.888056 17164 ts_tablet_manager.cc:1918] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:11.892935 17252 raft_consensus.cc:2726] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:11.898633 17147 raft_consensus.cc:504] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.888208 17249 raft_consensus.cc:482] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.891415 17146 tablet_bootstrap.cc:654] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.899752 17249 raft_consensus.cc:504] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.901546 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:11.901731 17249 leader_election.cc:288] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:11.902144 17041 raft_consensus.cc:3037] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:11.904763 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:11.906335 16972 leader_election.cc:341] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.906949 17164 log.cc:1196] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:11.907099 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:11.907827 17164 ts_tablet_manager.cc:1939] T 21fc7428f55041b6b939a8ca08574b90 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
W20260923 17:31:11.908582 16972 leader_election.cc:341] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.909004 16972 leader_election.cc:302] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.910070 17249 raft_consensus.cc:2726] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:11.912052 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 21fc7428f55041b6b939a8ca08574b90 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
I20260923 17:31:11.912808 16379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 723000dc175844e990740dc503b53ced
I20260923 17:31:11.912935 17041 raft_consensus.cc:2454] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:11.914088 16827 leader_election.cc:302] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:11.915509 17196 raft_consensus.cc:2781] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:11.915817 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:11.915896 17196 raft_consensus.cc:686] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:11.916684 17196 consensus_queue.cc:237] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.917755 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:11.917804 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:11.919261 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 723000dc175844e990740dc503b53ced because the tablet was not found. No further retry: Not found: Tablet not found: 723000dc175844e990740dc503b53ced
I20260923 17:31:11.919440 17249 raft_consensus.cc:482] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:11.919821 17249 raft_consensus.cc:504] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:11.921097 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 723000dc175844e990740dc503b53ced because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 723000dc175844e990740dc503b53ced already in progress: creating tablet
I20260923 17:31:11.921700 17249 leader_election.cc:288] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
W20260923 17:31:11.921821 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 723000dc175844e990740dc503b53ced because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 723000dc175844e990740dc503b53ced already in progress: creating tablet
I20260923 17:31:11.923725 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:11.925966 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:11.927299 16972 leader_election.cc:341] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:11.927968 16972 leader_election.cc:341] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.928414 16972 leader_election.cc:302] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:11.928848 17144 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:11.929245 17144 ts_tablet_manager.cc:1397] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.110s user 0.009s sys 0.003s
I20260923 17:31:11.929611 17249 raft_consensus.cc:2726] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:11.941121 17144 raft_consensus.cc:348] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.942037 17144 raft_consensus.cc:374] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.942271 17144 raft_consensus.cc:729] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.942603 17143 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:11.943089 17143 ts_tablet_manager.cc:1397] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.000s
I20260923 17:31:11.942891 17144 consensus_queue.cc:260] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.948611 17144 ts_tablet_manager.cc:1428] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.019s user 0.002s sys 0.004s
I20260923 17:31:11.954416 17144 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:11.948506 17143 raft_consensus.cc:348] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.955675 17146 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.956064 17146 ts_tablet_manager.cc:1397] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.076s user 0.013s sys 0.014s
I20260923 17:31:11.958108 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:11.958181 17146 raft_consensus.cc:348] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.958770 17146 raft_consensus.cc:374] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.958993 17146 raft_consensus.cc:729] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.959162 17143 raft_consensus.cc:374] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.959699 17143 raft_consensus.cc:729] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.959515 17146 consensus_queue.cc:260] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:11.960168 16826 leader_election.cc:341] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:11.961336 17146 ts_tablet_manager.cc:1428] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260923 17:31:11.962942 17146 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:11.963431 17143 consensus_queue.cc:260] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:11.966800 16379 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_49b13ac015354ceaa1f917b54abe16f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:11.968502 17146 tablet_bootstrap.cc:654] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:11.968557 16379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49b13ac015354ceaa1f917b54abe16f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:11.973344 17143 ts_tablet_manager.cc:1428] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260923 17:31:11.975382 17144 tablet_bootstrap.cc:654] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:11.988081 17147 leader_election.cc:288] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:11.988571 17146 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:11.989034 17146 ts_tablet_manager.cc:1397] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.000s
I20260923 17:31:11.991348 17146 raft_consensus.cc:348] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.992019 17146 raft_consensus.cc:374] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:11.992293 17146 raft_consensus.cc:729] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:11.992975 17146 consensus_queue.cc:260] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:11.991532 17143 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:11.996697 17146 ts_tablet_manager.cc:1428] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260923 17:31:12.004491 17144 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.004912 17144 ts_tablet_manager.cc:1397] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.009s
I20260923 17:31:12.006992 17144 raft_consensus.cc:348] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.007635 17144 raft_consensus.cc:374] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.007870 17144 raft_consensus.cc:729] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.008435 17146 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.014366 17146 tablet_bootstrap.cc:654] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.018738 17143 tablet_bootstrap.cc:654] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:12.020483 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8895cffac5945a19a5de7fba1319f22 consensus state has no leader
I20260923 17:31:12.020280 17144 consensus_queue.cc:260] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.021508 16388 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d65cfb2eb3648faab5869bba69cab3f from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:12.029129 17144 ts_tablet_manager.cc:1428] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.024s user 0.002s sys 0.003s
I20260923 17:31:12.036036 17144 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.040624 17146 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.041013 17146 ts_tablet_manager.cc:1397] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.003s
I20260923 17:31:12.042955 17146 raft_consensus.cc:348] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.043540 17146 raft_consensus.cc:374] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.043756 17146 raft_consensus.cc:729] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.044493 17146 consensus_queue.cc:260] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.046357 17146 ts_tablet_manager.cc:1428] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260923 17:31:12.047329 17146 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.054476 17144 tablet_bootstrap.cc:654] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.064785 17146 tablet_bootstrap.cc:654] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.068662 17144 tablet_bootstrap.cc:492] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.069031 17144 ts_tablet_manager.cc:1397] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.003s
I20260923 17:31:12.071012 17144 raft_consensus.cc:348] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.071609 17144 raft_consensus.cc:374] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.071823 17144 raft_consensus.cc:729] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.072541 17144 consensus_queue.cc:260] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.078297 17144 ts_tablet_manager.cc:1428] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260923 17:31:12.079178 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_afa52f4e296d4510a0ea5644aa44f26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.080761 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afa52f4e296d4510a0ea5644aa44f26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.087888 17146 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.088305 17146 ts_tablet_manager.cc:1397] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.004s
I20260923 17:31:12.090299 17146 raft_consensus.cc:348] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.090812 17146 raft_consensus.cc:374] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.091123 17146 raft_consensus.cc:729] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.091816 17146 consensus_queue.cc:260] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.092437 17144 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.097635 17144 tablet_bootstrap.cc:654] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.099139 17146 ts_tablet_manager.cc:1428] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260923 17:31:12.099898 17146 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.107722 17252 raft_consensus.cc:482] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.108023 17143 tablet_bootstrap.cc:492] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.108516 17143 ts_tablet_manager.cc:1397] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.117s user 0.014s sys 0.006s
I20260923 17:31:12.108163 17252 raft_consensus.cc:504] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.109167 17146 tablet_bootstrap.cc:654] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.109948 17252 leader_election.cc:288] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.112346 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fc7428f55041b6b939a8ca08574b90" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.112950 16893 raft_consensus.cc:2454] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:12.114663 16678 leader_election.cc:341] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:12.115650 16677 leader_election.cc:302] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:12.116633 17252 raft_consensus.cc:2781] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.116945 17252 raft_consensus.cc:482] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.117215 17252 raft_consensus.cc:3037] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.120446 17143 raft_consensus.cc:348] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.121171 17143 raft_consensus.cc:374] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.121444 17143 raft_consensus.cc:729] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.122030 17143 consensus_queue.cc:260] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.123870 17143 ts_tablet_manager.cc:1428] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.015s user 0.002s sys 0.001s
I20260923 17:31:12.124976 17252 raft_consensus.cc:504] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.128875 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2b330ec3646e4cecac1e58f4d2c2d79a" } modify_external_catalogs: true
I20260923 17:31:12.128469 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fc7428f55041b6b939a8ca08574b90" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:12.129531 16893 raft_consensus.cc:3037] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.129441 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2b330ec3646e4cecac1e58f4d2c2d79a" } modify_external_catalogs: true
I20260923 17:31:12.138034 17249 raft_consensus.cc:482] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.138509 17249 raft_consensus.cc:504] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.141530 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:12.144512 16972 leader_election.cc:341] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.145296 17249 leader_election.cc:288] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.145731 17249 raft_consensus.cc:482] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.146080 17249 raft_consensus.cc:504] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.147800 17249 leader_election.cc:288] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.149839 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.149914 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.150697 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:12.151366 16972 leader_election.cc:341] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.151757 17143 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:12.152035 16972 leader_election.cc:341] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.152825 16972 leader_election.cc:302] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.153591 17146 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.153955 17249 raft_consensus.cc:2726] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:12.155133 16972 leader_election.cc:341] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.155630 16972 leader_election.cc:302] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.156620 17249 raft_consensus.cc:2726] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.153985 17146 ts_tablet_manager.cc:1397] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.000s
I20260923 17:31:12.161451 16893 raft_consensus.cc:2454] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:12.162470 17146 raft_consensus.cc:348] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.163117 17146 raft_consensus.cc:374] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.163344 17146 raft_consensus.cc:729] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.163976 17146 consensus_queue.cc:260] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.166595 17285 raft_consensus.cc:482] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.167322 16677 leader_election.cc:302] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
W20260923 17:31:12.174412 16678 leader_election.cc:341] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:12.174861 17252 leader_election.cc:288] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.176090 17146 ts_tablet_manager.cc:1428] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260923 17:31:12.176702 17252 raft_consensus.cc:2781] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.176954 17146 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.178390 17144 tablet_bootstrap.cc:492] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.179265 17144 ts_tablet_manager.cc:1397] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.007s
I20260923 17:31:12.181510 17144 raft_consensus.cc:348] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.182206 17144 raft_consensus.cc:374] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.182482 17144 raft_consensus.cc:729] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.183483 17144 consensus_queue.cc:260] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.183950 17146 tablet_bootstrap.cc:654] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.175343 17285 raft_consensus.cc:504] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.190361 17143 tablet_bootstrap.cc:654] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.195875 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.195358 16380 catalog_manager.cc:5495] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:12.196511 17041 raft_consensus.cc:2454] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.196890 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_bdb104858f024c16a05173856e5c422e [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 2 got 0
I20260923 17:31:12.196940 16383 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 21cf533908d04a3a8ca087355528556f (table default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]). Expected version 1 got 0
I20260923 17:31:12.197767 16678 leader_election.cc:302] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:12.200037 17285 leader_election.cc:288] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.201560 16871 tablet_service.cc:1467] Processing CreateTablet for tablet ff22ffe24d424b1e8659579febfe1270 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.202667 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff22ffe24d424b1e8659579febfe1270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.184331 17252 raft_consensus.cc:686] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:12.204203 17252 consensus_queue.cc:237] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.204911 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 456273cf57314cb8a05c8e765543913f (DEFAULT_TABLE table=default.Table_c925434b48d5441eab0ddbc29bed59f5 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.205886 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 456273cf57314cb8a05c8e765543913f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.207273 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 456273cf57314cb8a05c8e765543913f (DEFAULT_TABLE table=default.Table_c925434b48d5441eab0ddbc29bed59f5 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.208962 16726 tablet_service.cc:1467] Processing CreateTablet for tablet ff22ffe24d424b1e8659579febfe1270 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.209431 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 456273cf57314cb8a05c8e765543913f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.210043 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff22ffe24d424b1e8659579febfe1270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.210623 17285 raft_consensus.cc:2781] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.210954 17285 raft_consensus.cc:482] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.211237 17285 raft_consensus.cc:3037] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.218230 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]). Expected version 1 got 0
I20260923 17:31:12.219236 16385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a4ca6aa0faa244799949fd44c5db72ed
W20260923 17:31:12.222618 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 55382.7us
I20260923 17:31:12.228677 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_cd593282564046359e7c7af93bc63eec [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 1 got 0
W20260923 17:31:12.230325 16387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 30d5c465348d4e68a5047d764bb802b5 consensus state has no leader
I20260923 17:31:12.231938 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ca6aa0faa244799949fd44c5db72ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:12.233250 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 456273cf57314cb8a05c8e765543913f (DEFAULT_TABLE table=default.Table_c925434b48d5441eab0ddbc29bed59f5 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.233593 17014 tablet_service.cc:1467] Processing CreateTablet for tablet ff22ffe24d424b1e8659579febfe1270 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:12.234385 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 456273cf57314cb8a05c8e765543913f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.234634 17163 tablet_replica.cc:330] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:12.235366 17163 raft_consensus.cc:2229] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:12.235816 17163 raft_consensus.cc:2258] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.237468 17163 ts_tablet_manager.cc:1905] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:12.240842 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ca6aa0faa244799949fd44c5db72ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:12.241801 17271 tablet_replica.cc:330] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:12.242563 17271 raft_consensus.cc:2229] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:12.243016 17271 raft_consensus.cc:2258] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.245196 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff22ffe24d424b1e8659579febfe1270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:12.251593 16387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_22a78bd0393140709e8a521e4480abf8" } new_table_name: "default.Table_b43616adac0b44a497e85e95855ff229" modify_external_catalogs: true
I20260923 17:31:12.256031 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.256278 17271 ts_tablet_manager.cc:1905] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:12.270020 16677 leader_election.cc:341] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.270902 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ca6aa0faa244799949fd44c5db72ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:12.271677 17164 tablet_replica.cc:330] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:12.272555 17164 raft_consensus.cc:2229] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:12.273209 17164 pending_rounds.cc:62] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:12.273398 17164 pending_rounds.cc:68] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596414901276672 op_type: NO_OP noop_request { }
I20260923 17:31:12.273737 17164 raft_consensus.cc:2866] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:12.274005 17164 raft_consensus.cc:2258] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.276357 17164 ts_tablet_manager.cc:1905] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:12.296602 17286 raft_consensus.cc:482] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.297178 17286 raft_consensus.cc:504] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.299168 17286 leader_election.cc:288] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.300457 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.300988 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.301977 17285 raft_consensus.cc:504] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:12.303020 16972 leader_election.cc:341] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:12.308734 16972 leader_election.cc:341] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.309132 16972 leader_election.cc:302] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.309887 17249 raft_consensus.cc:2726] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.312134 17286 raft_consensus.cc:482] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.312598 17286 raft_consensus.cc:504] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.314684 17249 raft_consensus.cc:482] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.315091 17249 raft_consensus.cc:504] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.316766 17249 leader_election.cc:288] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.316959 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:12.318322 16383 master_service.cc:946] ReplaceTablet: received request to replace tablet a4ca6aa0faa244799949fd44c5db72ed from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:12.318414 16677 leader_election.cc:341] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.319479 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:12.320619 16972 leader_election.cc:341] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.320508 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.321178 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:12.321703 16972 leader_election.cc:341] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:12.322664 16972 leader_election.cc:341] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.323014 16972 leader_election.cc:302] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.323679 17249 raft_consensus.cc:2726] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.324569 17286 leader_election.cc:288] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.324532 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.324924 17285 leader_election.cc:288] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.324592 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_8dccb00d41884ce5b86f1104790c363e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.326179 16972 leader_election.cc:341] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:12.326315 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dccb00d41884ce5b86f1104790c363e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.326635 16972 leader_election.cc:302] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.327339 17286 raft_consensus.cc:2726] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.335080 17146 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.335585 17146 ts_tablet_manager.cc:1397] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.159s user 0.024s sys 0.016s
I20260923 17:31:12.337643 17146 raft_consensus.cc:348] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.340932 17143 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.341390 17143 ts_tablet_manager.cc:1397] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.190s user 0.010s sys 0.000s
W20260923 17:31:12.328589 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 130910us
I20260923 17:31:12.344947 17143 raft_consensus.cc:348] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.345727 17143 raft_consensus.cc:374] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.346061 17143 raft_consensus.cc:729] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.346632 17143 consensus_queue.cc:260] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.349063 17143 ts_tablet_manager.cc:1428] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260923 17:31:12.353395 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5c465348d4e68a5047d764bb802b5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.353981 17041 raft_consensus.cc:3037] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.353992 17146 raft_consensus.cc:374] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.355243 17146 raft_consensus.cc:729] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.355847 17146 consensus_queue.cc:260] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.360486 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_17a12a23debd4da29e0cc2c01cd73d8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.362083 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17a12a23debd4da29e0cc2c01cd73d8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.368491 17041 raft_consensus.cc:2454] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:12.368352 16385 catalog_manager.cc:5495] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:12.369650 17143 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:12.369742 16385 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 21cf533908d04a3a8ca087355528556f (table default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]). Expected version 1 got 0
I20260923 17:31:12.371371 16678 leader_election.cc:302] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.371877 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 63808fe254ec4b1f818ed3198d5aea3d
I20260923 17:31:12.374600 17285 raft_consensus.cc:2781] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.375073 17285 raft_consensus.cc:686] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:12.375844 17143 tablet_bootstrap.cc:654] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.375768 17285 consensus_queue.cc:237] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.373070 17144 ts_tablet_manager.cc:1428] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.194s user 0.007s sys 0.010s
I20260923 17:31:12.378181 17144 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.381984 17146 ts_tablet_manager.cc:1428] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.046s user 0.002s sys 0.002s
I20260923 17:31:12.382606 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:12.382925 17144 tablet_bootstrap.cc:654] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.384326 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:12.384982 17146 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.340116 17271 ts_tablet_manager.cc:1918] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:12.386785 17271 log.cc:1196] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a4ca6aa0faa244799949fd44c5db72ed
I20260923 17:31:12.389221 17271 ts_tablet_manager.cc:1939] T a4ca6aa0faa244799949fd44c5db72ed P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
W20260923 17:31:12.389285 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 19) took 759 ms (client timeout 1000 ms). Trace:
I20260923 17:31:12.389503 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:12.391002 17146 tablet_bootstrap.cc:654] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.394297 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a4ca6aa0faa244799949fd44c5db72ed (table default.Table_2c9fdbd8c49e4d9dbe68c5157000786d [id=d0fff78552fb4121923c53b1a33d9994]) successfully deleted
W20260923 17:31:12.389768 16380 rpcz_store.cc:269] 0923 17:31:11.630075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:11.630225 (+ 150us) service_pool.cc:227] Handling call
0923 17:31:11.716551 (+ 86326us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:12.389205 (+672654us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:12.298137 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:12.298475 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:12.298499 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:12.298512 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:12.298776 (+ 264us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:12.298810 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:12.298832 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:12.298966 (+ 134us) tablet.cc:699] Row locks acquired
0923 17:31:12.298978 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:12.299083 (+ 105us) write_op.cc:270] Start()
0923 17:31:12.299163 (+ 80us) write_op.cc:276] Timestamp: P: 1790184672299011 usec, L: 0
0923 17:31:12.299179 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:12.299651 (+ 472us) log.cc:842] Serialized 535 byte log entry
0923 17:31:12.364846 (+ 65195us) op_driver.cc:444] REPLICATION: finished
0923 17:31:12.385218 (+ 20372us) write_op.cc:301] APPLY: starting
0923 17:31:12.385280 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:12.385374 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:12.385461 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:12.385630 (+ 169us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:12.385762 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:12.385805 (+ 43us) write_op.cc:312] APPLY: finished
0923 17:31:12.385994 (+ 189us) log.cc:842] Serialized 22 byte log entry
0923 17:31:12.386672 (+ 678us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:12.386737 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:12.386777 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:12.386944 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:12.386967 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":189320576,"child_traces":[["op",{"apply.queue_time_us":20239,"apply.run_cpu_time_us":1411,"apply.run_wall_time_us":1907,"num_ops":1,"prepare.queue_time_us":67485,"raft.queue_time_us":39788,"raft.run_cpu_time_us":19,"raft.run_wall_time_us":20,"replication_time_us":65601}]]}
I20260923 17:31:12.400032 17271 tablet_replica.cc:330] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:12.402352 17271 raft_consensus.cc:2229] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:12.404443 17271 raft_consensus.cc:2258] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.406742 17144 tablet_bootstrap.cc:492] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.409307 17144 ts_tablet_manager.cc:1397] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.005s
I20260923 17:31:12.412104 17144 raft_consensus.cc:348] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.413765 17271 ts_tablet_manager.cc:1905] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:12.415334 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3d26a0116b2b4710a7482f575d4fd172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.417857 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d26a0116b2b4710a7482f575d4fd172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.413039 17294 raft_consensus.cc:482] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.423418 17294 raft_consensus.cc:504] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.425748 17294 leader_election.cc:288] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.427264 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:12.428804 16677 leader_election.cc:341] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:31:12.430356 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.430955 17041 raft_consensus.cc:2454] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.432053 16678 leader_election.cc:302] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.436441 17294 raft_consensus.cc:2781] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.436720 17294 raft_consensus.cc:482] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.436986 17294 raft_consensus.cc:3037] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:12.445633 16387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ceb787c0863847289400e62718d218a1 consensus state has no leader
I20260923 17:31:12.450513 17144 raft_consensus.cc:374] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.451259 16387 master_service.cc:946] ReplaceTablet: received request to replace tablet 29d44ee57faf4a1d99e077fd781a519c from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:12.450805 17144 raft_consensus.cc:729] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.456478 17144 consensus_queue.cc:260] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.457767 17294 raft_consensus.cc:504] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.460091 17164 ts_tablet_manager.cc:1918] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:12.465584 16384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_f03a6d12e8cb4f5298b3da28260ac01f" } modify_external_catalogs: true
I20260923 17:31:12.466202 16384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_f03a6d12e8cb4f5298b3da28260ac01f" } modify_external_catalogs: true
I20260923 17:31:12.466461 17144 ts_tablet_manager.cc:1428] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.056s user 0.004s sys 0.000s
I20260923 17:31:12.469477 16379 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_162de7ee3faf4ec3916e0cd2bcef3d45" } new_table_name: "default.Table_6bb854c7f23b41c284c4bb2b753fe3ae" modify_external_catalogs: true
I20260923 17:31:12.472373 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:12.473520 16677 leader_election.cc:341] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:31:12.474359 17143 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.474728 17143 ts_tablet_manager.cc:1397] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.105s user 0.019s sys 0.033s
I20260923 17:31:12.476877 17144 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.478046 17294 leader_election.cc:288] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.488816 17143 raft_consensus.cc:348] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.489493 17143 raft_consensus.cc:374] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.489724 17143 raft_consensus.cc:729] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.490275 17143 consensus_queue.cc:260] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.491977 17143 ts_tablet_manager.cc:1428] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260923 17:31:12.493875 17144 tablet_bootstrap.cc:654] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.510545 17143 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:12.510054 17271 ts_tablet_manager.cc:1918] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:12.511202 17271 log.cc:1196] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/63808fe254ec4b1f818ed3198d5aea3d
I20260923 17:31:12.512337 17271 ts_tablet_manager.cc:1939] T 63808fe254ec4b1f818ed3198d5aea3d P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:12.533103 17294 raft_consensus.cc:482] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.533275 17143 tablet_bootstrap.cc:654] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.533525 17294 raft_consensus.cc:504] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.535358 17294 leader_election.cc:288] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.535738 17147 raft_consensus.cc:482] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.536350 17147 raft_consensus.cc:504] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.537959 17294 raft_consensus.cc:482] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.538407 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.538314 17294 raft_consensus.cc:504] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.539017 16893 raft_consensus.cc:2454] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.541886 16677 leader_election.cc:302] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:12.453490 17163 ts_tablet_manager.cc:1918] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:12.542912 17285 raft_consensus.cc:2781] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.544152 17163 log.cc:1196] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a4ca6aa0faa244799949fd44c5db72ed
I20260923 17:31:12.544269 17285 raft_consensus.cc:482] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.545073 17285 raft_consensus.cc:3037] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.545145 17164 log.cc:1196] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a4ca6aa0faa244799949fd44c5db72ed
I20260923 17:31:12.544001 17249 raft_consensus.cc:482] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.545849 17164 ts_tablet_manager.cc:1939] T a4ca6aa0faa244799949fd44c5db72ed P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:12.545796 17249 raft_consensus.cc:504] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.547691 17249 leader_election.cc:288] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.460299 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63808fe254ec4b1f818ed3198d5aea3d" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.461447 17146 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.548352 17164 tablet_replica.cc:330] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:12.548581 17146 ts_tablet_manager.cc:1397] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.164s user 0.008s sys 0.000s
I20260923 17:31:12.543696 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.548753 17041 raft_consensus.cc:3037] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.549147 16744 raft_consensus.cc:2454] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:12.549513 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.550740 17146 raft_consensus.cc:348] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:12.551167 16972 leader_election.cc:341] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.551395 17146 raft_consensus.cc:374] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.551651 17146 raft_consensus.cc:729] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.471455 17286 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.552937 16826 leader_election.cc:302] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:12.552409 17286 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.554001 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a4ca6aa0faa244799949fd44c5db72ed (table default.Table_2c9fdbd8c49e4d9dbe68c5157000786d [id=d0fff78552fb4121923c53b1a33d9994]) successfully deleted
I20260923 17:31:12.541898 17294 leader_election.cc:288] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.554761 17286 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.538854 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.555327 17039 raft_consensus.cc:2454] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.540777 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.555421 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_0cef567b57404c0ab86a30bf79964ef2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:12.556077 17040 raft_consensus.cc:2454] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:12.541255 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.552181 17146 consensus_queue.cc:260] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.557008 17038 raft_consensus.cc:2379] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:12.556938 17041 raft_consensus.cc:2454] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
W20260923 17:31:12.557303 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cef567b57404c0ab86a30bf79964ef2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.558421 17146 ts_tablet_manager.cc:1428] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260923 17:31:12.558519 16678 leader_election.cc:302] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.558991 17164 raft_consensus.cc:2229] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:12.559437 17164 raft_consensus.cc:2258] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.559448 17294 raft_consensus.cc:2781] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.559875 17146 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.561088 17164 ts_tablet_manager.cc:1905] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:12.562624 17294 raft_consensus.cc:686] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:12.563377 17294 consensus_queue.cc:237] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:12.567853 17146 tablet_bootstrap.cc:654] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.567761 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cf533908d04a3a8ca087355528556f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.569564 17285 raft_consensus.cc:504] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.544816 17163 ts_tablet_manager.cc:1939] T a4ca6aa0faa244799949fd44c5db72ed P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:12.576972 17146 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.580693 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.581025 17163 tablet_replica.cc:330] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:12.582367 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a4ca6aa0faa244799949fd44c5db72ed (table default.Table_2c9fdbd8c49e4d9dbe68c5157000786d [id=d0fff78552fb4121923c53b1a33d9994]) successfully deleted
I20260923 17:31:12.583019 17146 ts_tablet_manager.cc:1397] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.006s
W20260923 17:31:12.583688 16972 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.583853 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]) successfully deleted
I20260923 17:31:12.538435 17147 leader_election.cc:288] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.585121 17146 raft_consensus.cc:348] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.585780 17146 raft_consensus.cc:374] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.586505 17147 raft_consensus.cc:2781] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.586928 17147 raft_consensus.cc:482] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.584412 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.586926 16388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_be66ae961c4342319e013d42905625d1" } modify_external_catalogs: true
I20260923 17:31:12.587469 16388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_be66ae961c4342319e013d42905625d1" } modify_external_catalogs: true
I20260923 17:31:12.588203 17147 raft_consensus.cc:3037] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:12.589054 16972 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.586010 17146 raft_consensus.cc:729] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.589478 16972 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.590219 17286 raft_consensus.cc:2726] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.590137 17146 consensus_queue.cc:260] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.592454 17143 tablet_bootstrap.cc:492] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.592844 17143 ts_tablet_manager.cc:1397] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.082s user 0.010s sys 0.000s
I20260923 17:31:12.595112 17143 raft_consensus.cc:348] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.595583 17143 raft_consensus.cc:374] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.596127 17143 raft_consensus.cc:729] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.596709 17143 consensus_queue.cc:260] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.598526 17143 ts_tablet_manager.cc:1428] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260923 17:31:12.599263 17143 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:12.610160 17144 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.609876 17164 ts_tablet_manager.cc:1918] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:12.610556 17144 ts_tablet_manager.cc:1397] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.004s
I20260923 17:31:12.610642 17164 log.cc:1196] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/63808fe254ec4b1f818ed3198d5aea3d
I20260923 17:31:12.612062 17285 leader_election.cc:288] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.613590 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.613278 17147 raft_consensus.cc:504] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.615329 17147 leader_election.cc:288] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.615290 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.615937 17041 raft_consensus.cc:3037] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.616971 17144 raft_consensus.cc:348] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.617587 17144 raft_consensus.cc:374] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260923 17:31:12.617535 16972 leader_election.cc:341] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.617798 17144 raft_consensus.cc:729] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.617940 16972 leader_election.cc:302] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.618393 17144 consensus_queue.cc:260] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.618953 17143 tablet_bootstrap.cc:654] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.622048 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.622593 17038 raft_consensus.cc:3037] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.624812 16677 leader_election.cc:302] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:12.625510 17285 raft_consensus.cc:3037] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.627637 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d22f0f9d7e6a46a08ca9f411cc2c658b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:12.628268 16893 raft_consensus.cc:3037] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.630244 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8879dee684214c6dac3f9d6642a6d97d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:12.630753 16744 raft_consensus.cc:3037] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.631779 17038 raft_consensus.cc:2454] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:12.637295 16744 raft_consensus.cc:2454] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:12.637392 16383 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b9b734b250d4920abbafe76f52ec540 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:12.638397 17144 ts_tablet_manager.cc:1428] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260923 17:31:12.638154 16826 leader_election.cc:302] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:12.638962 17303 raft_consensus.cc:2781] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.639370 17303 raft_consensus.cc:686] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:12.640096 17303 consensus_queue.cc:237] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.644953 17163 raft_consensus.cc:2229] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:12.645432 16384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:12.646479 17144 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.644829 17285 raft_consensus.cc:2726] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.647406 17163 pending_rounds.cc:62] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:12.647295 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:12.647634 17163 pending_rounds.cc:68] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596419006472192 op_type: NO_OP noop_request { }
I20260923 17:31:12.649537 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:12.648052 17163 raft_consensus.cc:2866] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:12.650054 17163 raft_consensus.cc:2258] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.649856 16387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_48c7725ec6034f21b65906e2c8c0f02c" } new_table_name: "default.Table_5fbe014655974cf987e980ebcb49dbc9" modify_external_catalogs: true
W20260923 17:31:12.651501 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 3b9b734b250d4920abbafe76f52ec540 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b9b734b250d4920abbafe76f52ec540 already in progress: creating tablet
I20260923 17:31:12.652480 17163 ts_tablet_manager.cc:1905] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:12.661563 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 44067.8us
I20260923 17:31:12.662146 17249 raft_consensus.cc:2726] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.665145 17249 consensus_queue.cc:1035] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:12.674062 17249 raft_consensus.cc:482] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.674467 17249 raft_consensus.cc:504] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.680150 17249 leader_election.cc:288] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.681350 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:12.681653 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9b734b250d4920abbafe76f52ec540" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:12.682931 16972 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.683032 16893 raft_consensus.cc:2454] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:12.683739 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9b734b250d4920abbafe76f52ec540" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.684417 16677 leader_election.cc:302] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
W20260923 17:31:12.684899 16972 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.685230 16972 leader_election.cc:302] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.686353 17285 raft_consensus.cc:2781] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.687041 17285 raft_consensus.cc:686] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:12.687836 17285 consensus_queue.cc:237] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.692375 17249 raft_consensus.cc:2726] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.692190 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_dbbb33fb20264319a1ace8d6d1553144"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:12.695065 17144 tablet_bootstrap.cc:654] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:12.695729 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbbb33fb20264319a1ace8d6d1553144 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.697688 17143 tablet_bootstrap.cc:492] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.698246 17143 ts_tablet_manager.cc:1397] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.000s
I20260923 17:31:12.701097 17146 ts_tablet_manager.cc:1428] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.118s user 0.005s sys 0.000s
I20260923 17:31:12.705583 17041 raft_consensus.cc:2454] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:12.712841 17164 ts_tablet_manager.cc:1939] T 63808fe254ec4b1f818ed3198d5aea3d P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:12.715438 17164 tablet_replica.cc:330] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:12.716310 17164 raft_consensus.cc:2229] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:12.716744 17164 raft_consensus.cc:2258] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:12.712034 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.718295 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.718953 17164 ts_tablet_manager.cc:1905] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:12.721274 17143 raft_consensus.cc:348] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.724864 17146 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:12.729602 17146 tablet_bootstrap.cc:654] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.739744 17164 ts_tablet_manager.cc:1918] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:12.726900 17303 raft_consensus.cc:482] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.740276 17164 log.cc:1196] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:12.740550 17143 raft_consensus.cc:374] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.741032 17164 ts_tablet_manager.cc:1939] T 3b9b734b250d4920abbafe76f52ec540 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:12.740876 17143 raft_consensus.cc:729] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.740653 17303 raft_consensus.cc:504] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.742707 17303 leader_election.cc:288] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.743752 17143 consensus_queue.cc:260] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:12.762336 16373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260923 17:31:12.764072 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.767791 17041 raft_consensus.cc:2454] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:12.768898 16827 leader_election.cc:302] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:12.773608 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]) successfully deleted
I20260923 17:31:12.775965 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 3b9b734b250d4920abbafe76f52ec540 (table default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]) successfully deleted
I20260923 17:31:12.776888 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_8696ac8b31fa47da987d4daf0e56100f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:12.780567 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.782130 17314 consensus_queue.cc:1035] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:12.783483 16826 leader_election.cc:341] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.786055 17317 tablet.cc:1722] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:12.801503 17314 raft_consensus.cc:482] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.802138 17314 raft_consensus.cc:504] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:12.802739 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8696ac8b31fa47da987d4daf0e56100f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.806644 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:12.808025 16972 leader_election.cc:341] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.809042 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.809962 17314 leader_election.cc:288] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
W20260923 17:31:12.810413 16972 leader_election.cc:341] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.810789 16972 leader_election.cc:302] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.811445 17314 raft_consensus.cc:2726] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:12.811650 17310 log.cc:924] Time spent T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Append to log took a long time: real 0.109s user 0.001s sys 0.000s
W20260923 17:31:12.828786 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260923 17:31:12.831372 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e2fa2e2d002746c8ba2b0dcf45372aa8 consensus state has no leader
W20260923 17:31:12.845332 16827 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 76530.4us
I20260923 17:31:12.847898 17303 raft_consensus.cc:2781] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.848341 17303 raft_consensus.cc:482] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.848631 17303 raft_consensus.cc:3037] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.839852 16386 master_service.cc:946] ReplaceTablet: received request to replace tablet ff22ffe24d424b1e8659579febfe1270 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:12.852689 17285 raft_consensus.cc:482] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.853196 17285 raft_consensus.cc:504] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.855114 17303 raft_consensus.cc:504] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.856711 17294 raft_consensus.cc:482] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.856819 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.857147 17294 raft_consensus.cc:504] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.858902 17294 leader_election.cc:288] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.859529 17303 leader_election.cc:288] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:12.860098 16677 leader_election.cc:341] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.860584 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:12.862227 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.862846 17038 raft_consensus.cc:2454] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.865182 16678 leader_election.cc:302] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:12.866226 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.867149 17294 raft_consensus.cc:2781] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.867422 17294 raft_consensus.cc:482] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.867668 17294 raft_consensus.cc:3037] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.873436 17144 tablet_bootstrap.cc:492] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:12.855125 17285 leader_election.cc:288] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.874004 17144 ts_tablet_manager.cc:1397] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.228s user 0.011s sys 0.000s
W20260923 17:31:12.874626 16826 leader_election.cc:341] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:12.875696 16677 leader_election.cc:341] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.878131 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.878700 17038 raft_consensus.cc:3037] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.879418 17294 raft_consensus.cc:504] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.881238 17294 leader_election.cc:288] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.883270 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:12.883823 17040 raft_consensus.cc:3037] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.888957 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:12.890537 17143 ts_tablet_manager.cc:1428] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.192s user 0.007s sys 0.000s
I20260923 17:31:12.890918 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_49b21ea5e5384eb8ab2d4942fabcf5a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:12.891459 17143 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:12.892951 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49b21ea5e5384eb8ab2d4942fabcf5a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.896240 17303 consensus_queue.cc:1035] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:12.897043 16384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_46a8dbb79853475b8593a2de25a1bc3b" } modify_external_catalogs: true
I20260923 17:31:12.897698 16384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_46a8dbb79853475b8593a2de25a1bc3b" } modify_external_catalogs: true
W20260923 17:31:12.898352 16677 leader_election.cc:341] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.901026 17319 tablet.cc:1722] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:12.905651 17143 tablet_bootstrap.cc:654] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.908530 17038 raft_consensus.cc:2454] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:12.910329 16827 leader_election.cc:302] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217
I20260923 17:31:12.911629 17303 raft_consensus.cc:2781] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.912060 17303 raft_consensus.cc:686] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:12.912774 17303 consensus_queue.cc:237] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.914105 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:12.914937 17041 raft_consensus.cc:2454] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:12.923491 17163 ts_tablet_manager.cc:1918] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:12.923836 17163 log.cc:1196] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/63808fe254ec4b1f818ed3198d5aea3d
I20260923 17:31:12.924536 17163 ts_tablet_manager.cc:1939] T 63808fe254ec4b1f818ed3198d5aea3d P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:12.925457 17143 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.925897 17143 ts_tablet_manager.cc:1397] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.000s
I20260923 17:31:12.928258 17143 raft_consensus.cc:348] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.929112 17143 raft_consensus.cc:374] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.929375 17143 raft_consensus.cc:729] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.930133 17143 consensus_queue.cc:260] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.932389 17143 ts_tablet_manager.cc:1428] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260923 17:31:12.932595 17144 raft_consensus.cc:348] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.933211 17143 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:12.933264 17144 raft_consensus.cc:374] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.933508 17144 raft_consensus.cc:729] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.939523 17143 tablet_bootstrap.cc:654] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.934067 17144 consensus_queue.cc:260] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.942683 17144 ts_tablet_manager.cc:1428] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.068s user 0.005s sys 0.000s
I20260923 17:31:12.950448 17143 tablet_bootstrap.cc:492] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:12.942965 17314 raft_consensus.cc:482] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.950851 17143 ts_tablet_manager.cc:1397] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260923 17:31:12.951160 17314 raft_consensus.cc:504] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.953086 17314 leader_election.cc:288] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:12.953243 17143 raft_consensus.cc:348] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.953871 17143 raft_consensus.cc:374] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.954286 17143 raft_consensus.cc:729] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.954883 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:12.954844 17143 consensus_queue.cc:260] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.955353 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:12.956465 16972 leader_election.cc:341] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.957154 17143 ts_tablet_manager.cc:1428] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260923 17:31:12.957118 16972 leader_election.cc:341] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:12.957469 16972 leader_election.cc:302] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.957978 17143 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:12.958887 17314 raft_consensus.cc:2726] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:12.960573 16678 leader_election.cc:302] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.961558 17147 raft_consensus.cc:482] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:12.961607 17294 raft_consensus.cc:2781] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.961966 17147 raft_consensus.cc:504] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.962517 17294 raft_consensus.cc:482] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.962810 17294 raft_consensus.cc:3037] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.963454 17146 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:12.963778 17147 leader_election.cc:288] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:12.963877 17146 ts_tablet_manager.cc:1397] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.239s user 0.006s sys 0.005s
I20260923 17:31:12.899169 17321 mvcc.cc:205] Tried to move back new op lower bound from 7332596419404730368 to 7332596414890721280. Current Snapshot: MvccSnapshot[applied={T|T < 7332596419404730368}]
I20260923 17:31:12.965307 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d44ee57faf4a1d99e077fd781a519c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:12.965900 16744 raft_consensus.cc:2454] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
W20260923 17:31:12.966240 16827 leader_election.cc:341] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:12.966958 17143 tablet_bootstrap.cc:654] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:12.968909 17326 tablet.cc:1722] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:12.970131 17040 raft_consensus.cc:2454] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:12.972823 17144 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:12.978755 16678 leader_election.cc:302] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:12.972613 17146 raft_consensus.cc:348] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.979473 17146 raft_consensus.cc:374] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:12.979800 17146 raft_consensus.cc:729] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:12.980533 17285 raft_consensus.cc:2781] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:12.980948 17285 raft_consensus.cc:686] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:12.981210 16826 leader_election.cc:302] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:12.981787 17285 consensus_queue.cc:237] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.982700 17146 consensus_queue.cc:260] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:12.984539 17147 raft_consensus.cc:2781] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:12.984900 17147 raft_consensus.cc:482] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:12.985195 17147 raft_consensus.cc:3037] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:12.985358 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_e69cd0d31b464f32a2c86b830f18b7df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:12.986963 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e69cd0d31b464f32a2c86b830f18b7df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:12.988603 17144 tablet_bootstrap.cc:654] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.000478 17294 raft_consensus.cc:504] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.001719 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 85eba2aac8d84defa8ce4081d447bc63 (DEFAULT_TABLE table=default.Table_ce78da2e733545078e9762632b134528 [id=6265c59de8be492aa44b9eb0f0f10c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.002883 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85eba2aac8d84defa8ce4081d447bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.006016 17016 tablet_service.cc:1467] Processing CreateTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (DEFAULT_TABLE table=default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.007141 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389ecb50e0c34b4d92b5296bb17f4ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.011693 17146 ts_tablet_manager.cc:1428] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.048s user 0.003s sys 0.003s
I20260923 17:31:13.014629 17018 tablet_service.cc:1467] Processing CreateTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 (DEFAULT_TABLE table=default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.015735 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2beb8360d54efeb2fa74c005d011d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.016882 17021 tablet_service.cc:1467] Processing CreateTablet for tablet e818f6192de6472cb27747c0e3a1f128 (DEFAULT_TABLE table=default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.017810 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e818f6192de6472cb27747c0e3a1f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.017990 17146 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.023557 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_bdb104858f024c16a05173856e5c422e [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 2 got 0
I20260923 17:31:13.024478 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:13.025624 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_516ec99807654f5385fc92422c51562a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:13.028146 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_516ec99807654f5385fc92422c51562a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.032130 17143 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:13.032680 17143 ts_tablet_manager.cc:1397] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.014s
I20260923 17:31:13.035526 17143 raft_consensus.cc:348] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.037678 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7]) successfully deleted
I20260923 17:31:13.038501 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet ceb787c0863847289400e62718d218a1 (table default.Table_b43616adac0b44a497e85e95855ff229 [id=f3dfb81ee57147d79f9c67dbbaf451b1]). Expected version 1 got 0
W20260923 17:31:13.039177 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 3b9b734b250d4920abbafe76f52ec540 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b9b734b250d4920abbafe76f52ec540 already in progress: creating tablet
I20260923 17:31:13.039714 17147 raft_consensus.cc:504] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.040886 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]). Expected version 1 got 0
I20260923 17:31:13.042250 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:13.043332 17147 leader_election.cc:288] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.044533 16380 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:13.049499 17146 tablet_bootstrap.cc:654] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.050408 17143 raft_consensus.cc:374] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.050776 17143 raft_consensus.cc:729] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:13.053957 16827 leader_election.cc:341] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:13.055363 17143 consensus_queue.cc:260] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.057444 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet f124169d87c240a5bd544d3b6693323f (table default.Table_48c7725ec6034f21b65906e2c8c0f02c [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 1 got 0
I20260923 17:31:13.058151 17143 ts_tablet_manager.cc:1428] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260923 17:31:13.058187 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 21cf533908d04a3a8ca087355528556f (table default.Table_162de7ee3faf4ec3916e0cd2bcef3d45 [id=7f9bdab909c047f09894500c06577ddd]). Expected version 1 got 0
I20260923 17:31:13.058930 16381 catalog_manager.cc:5495] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:13.060026 17314 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.060473 17314 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.060469 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:13.062222 17314 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
W20260923 17:31:13.063202 16387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f124169d87c240a5bd544d3b6693323f consensus state has no leader
I20260923 17:31:13.063948 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.067514 17143 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:13.071007 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 24) took 813 ms (client timeout 1000 ms). Trace:
I20260923 17:31:13.074357 17143 tablet_bootstrap.cc:654] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:13.079660 16972 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.085412 17017 tablet_service.cc:1467] Processing CreateTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed (DEFAULT_TABLE table=default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.086438 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a182b6ced72428e9638465f0cc4e7ed. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:13.071394 16385 rpcz_store.cc:269] 0923 17:31:12.257786 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:12.258242 (+ 456us) service_pool.cc:227] Handling call
0923 17:31:13.070901 (+812659us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:12.476598 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:12.476919 (+ 321us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:12.476942 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:12.476954 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:12.477197 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:12.477233 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:12.477254 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:12.477396 (+ 142us) tablet.cc:699] Row locks acquired
0923 17:31:12.477410 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:12.477514 (+ 104us) write_op.cc:270] Start()
0923 17:31:12.477600 (+ 86us) write_op.cc:276] Timestamp: P: 1790184672477468 usec, L: 0
0923 17:31:12.477616 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:12.478038 (+ 422us) log.cc:842] Serialized 535 byte log entry
0923 17:31:12.493027 (+ 14989us) op_driver.cc:444] REPLICATION: finished
0923 17:31:12.494120 (+ 1093us) write_op.cc:301] APPLY: starting
0923 17:31:12.494169 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:12.494253 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:12.494335 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:12.494476 (+ 141us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:12.494609 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:12.494648 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:12.494822 (+ 174us) log.cc:842] Serialized 22 byte log entry
0923 17:31:12.494945 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:12.494993 (+ 48us) write_op.cc:454] Released schema lock
0923 17:31:12.495022 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:12.495175 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:12.495196 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":949,"apply.run_cpu_time_us":1220,"apply.run_wall_time_us":1220,"num_ops":1,"prepare.queue_time_us":80466,"prepare.run_cpu_time_us":1734,"prepare.run_wall_time_us":10471,"raft.queue_time_us":73,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":15334}]]}
I20260923 17:31:13.089439 17020 tablet_service.cc:1467] Processing CreateTablet for tablet d2796fcb72f5497a92208b40224739e9 (DEFAULT_TABLE table=default.Table_2d35179a997541eda78f604ede819340 [id=17f994d2e127410685fffe47828a215b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.091533 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2796fcb72f5497a92208b40224739e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.093528 17014 tablet_service.cc:1467] Processing CreateTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 (DEFAULT_TABLE table=default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.094462 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f62eadc6a94aedb4a8164c545cda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.003707 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (DEFAULT_TABLE table=default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.005077 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 85eba2aac8d84defa8ce4081d447bc63 (DEFAULT_TABLE table=default.Table_ce78da2e733545078e9762632b134528 [id=6265c59de8be492aa44b9eb0f0f10c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.010579 16717 tablet_service.cc:1467] Processing CreateTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 (DEFAULT_TABLE table=default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.011708 16721 tablet_service.cc:1467] Processing CreateTablet for tablet e818f6192de6472cb27747c0e3a1f128 (DEFAULT_TABLE table=default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.021097 16725 tablet_service.cc:1467] Processing CreateTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 (DEFAULT_TABLE table=default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.023770 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed (DEFAULT_TABLE table=default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.099342 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389ecb50e0c34b4d92b5296bb17f4ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.028685 16720 tablet_service.cc:1467] Processing CreateTablet for tablet d2796fcb72f5497a92208b40224739e9 (DEFAULT_TABLE table=default.Table_2d35179a997541eda78f604ede819340 [id=17f994d2e127410685fffe47828a215b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.065505 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:13.101143 16972 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.101696 16972 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:13.104883 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2beb8360d54efeb2fa74c005d011d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.104862 17314 raft_consensus.cc:2726] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.109933 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e818f6192de6472cb27747c0e3a1f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.117158 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f62eadc6a94aedb4a8164c545cda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.122566 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2796fcb72f5497a92208b40224739e9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:13.144479 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 162056us
I20260923 17:31:13.148075 17146 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:13.148496 17146 ts_tablet_manager.cc:1397] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.131s user 0.012s sys 0.002s
I20260923 17:31:13.150545 17146 raft_consensus.cc:348] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.151165 17146 raft_consensus.cc:374] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.151379 17146 raft_consensus.cc:729] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.158629 16379 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:13.161047 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_2402388e897844ae8d054926cdd0aa80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:13.162719 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2402388e897844ae8d054926cdd0aa80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.166114 17332 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260923 17:31:13.167755 16379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.166646 17332 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.182484 17332 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.192589 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85eba2aac8d84defa8ce4081d447bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.198263 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a182b6ced72428e9638465f0cc4e7ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.203852 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 85eba2aac8d84defa8ce4081d447bc63 (DEFAULT_TABLE table=default.Table_ce78da2e733545078e9762632b134528 [id=6265c59de8be492aa44b9eb0f0f10c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.205241 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85eba2aac8d84defa8ce4081d447bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.238852 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d44ee57faf4a1d99e077fd781a519c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:13.239533 16743 raft_consensus.cc:3037] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.243376 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (DEFAULT_TABLE table=default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.244693 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389ecb50e0c34b4d92b5296bb17f4ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.245453 16870 tablet_service.cc:1467] Processing CreateTablet for tablet e818f6192de6472cb27747c0e3a1f128 (DEFAULT_TABLE table=default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.246584 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e818f6192de6472cb27747c0e3a1f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.246889 16743 raft_consensus.cc:2454] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:13.248842 16893 raft_consensus.cc:3037] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.249003 16866 tablet_service.cc:1467] Processing CreateTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 (DEFAULT_TABLE table=default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.250027 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6f62eadc6a94aedb4a8164c545cda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.253153 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed (DEFAULT_TABLE table=default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.254230 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a182b6ced72428e9638465f0cc4e7ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.243453 16867 tablet_service.cc:1467] Processing CreateTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 (DEFAULT_TABLE table=default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.256456 17294 leader_election.cc:288] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.257664 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2beb8360d54efeb2fa74c005d011d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.264453 17292 consensus_queue.cc:1035] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:13.266000 17292 consensus_queue.cc:1035] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:13.269418 17336 log.cc:924] Time spent T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260923 17:31:13.272171 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:13.285027 16677 leader_election.cc:341] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.286609 16873 tablet_service.cc:1467] Processing CreateTablet for tablet d2796fcb72f5497a92208b40224739e9 (DEFAULT_TABLE table=default.Table_2d35179a997541eda78f604ede819340 [id=17f994d2e127410685fffe47828a215b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:13.287586 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 63808fe254ec4b1f818ed3198d5aea3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:13.288636 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:13.288978 16826 leader_election.cc:302] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:13.289554 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2796fcb72f5497a92208b40224739e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:13.305567 16384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:13.307019 16388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:13.310958 17332 raft_consensus.cc:2781] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:13.311465 17332 raft_consensus.cc:686] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:13.312424 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_8dccb00d41884ce5b86f1104790c363e" } new_table_name: "default.Table_7a8e6e0a311345c1bb25f336834dc7f4" modify_external_catalogs: true
W20260923 17:31:13.316293 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 20) took 908 ms (client timeout 1000 ms). Trace:
I20260923 17:31:13.321208 16384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_49b21ea5e5384eb8ab2d4942fabcf5a2" } new_table_name: "default.Table_95f3f4adfca44bc791a565a2d16257f4" modify_external_catalogs: true
W20260923 17:31:13.326273 16380 rpcz_store.cc:269] 0923 17:31:12.407441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:12.407629 (+ 188us) service_pool.cc:227] Handling call
0923 17:31:13.316190 (+908561us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":15}
I20260923 17:31:13.327381 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:13.328763 16386 catalog_manager.cc:6565] ReplaceTablet: tablet ff22ffe24d424b1e8659579febfe1270 deleted and replaced by tablet 051f2e478a324df98647d90107d55533
I20260923 17:31:13.329471 16719 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:13.330107 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:13.330788 16722 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:13.312242 17332 consensus_queue.cc:237] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.333875 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because the tablet was not found. No further retry: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed
W20260923 17:31:13.334550 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 389ecb50e0c34b4d92b5296bb17f4ab1 already in progress: creating tablet
I20260923 17:31:13.335539 17143 tablet_bootstrap.cc:492] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:13.336058 17143 ts_tablet_manager.cc:1397] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.269s user 0.081s sys 0.078s
I20260923 17:31:13.337309 17144 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:13.337698 17144 ts_tablet_manager.cc:1397] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.365s user 0.026s sys 0.013s
I20260923 17:31:13.336196 17294 raft_consensus.cc:482] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.339610 17294 raft_consensus.cc:504] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.341877 17294 leader_election.cc:288] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:13.343551 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff22ffe24d424b1e8659579febfe1270 already in progress: creating tablet
I20260923 17:31:13.344280 16386 master_service.cc:946] ReplaceTablet: received request to replace tablet 0721fa178c0347598351b9d3512089b8 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:13.345323 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:13.351724 17144 raft_consensus.cc:348] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.352371 17144 raft_consensus.cc:374] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.352591 17144 raft_consensus.cc:729] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.353022 17144 consensus_queue.cc:260] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.354770 17144 ts_tablet_manager.cc:1428] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260923 17:31:13.355343 17340 tablet_replica.cc:330] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:13.185171 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:13.363348 17143 raft_consensus.cc:348] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.364070 17143 raft_consensus.cc:374] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.364341 17143 raft_consensus.cc:729] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.365043 17144 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:13.365854 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.370069 17144 tablet_bootstrap.cc:654] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.371882 17340 raft_consensus.cc:2229] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:13.372852 17340 pending_rounds.cc:62] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Trying to abort 1 pending ops.
I20260923 17:31:13.373073 17340 pending_rounds.cc:68] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596417357897728 op_type: NO_OP noop_request { }
I20260923 17:31:13.374403 17144 tablet_bootstrap.cc:492] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
W20260923 17:31:13.334848 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 45937.3us
I20260923 17:31:13.374744 17144 ts_tablet_manager.cc:1397] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260923 17:31:13.373937 17143 consensus_queue.cc:260] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.377305 17144 raft_consensus.cc:348] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.377915 17144 raft_consensus.cc:374] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.378145 17144 raft_consensus.cc:729] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.378769 17144 consensus_queue.cc:260] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.373593 17340 raft_consensus.cc:2866] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:13.379596 17143 ts_tablet_manager.cc:1428] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
W20260923 17:31:13.382066 16374 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.189:45151, user_credentials={real_user=slave}} blocked reactor thread for 49676.1us
W20260923 17:31:13.386235 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 63808fe254ec4b1f818ed3198d5aea3d because the tablet was not found. No further retry: Not found: Tablet not found: 63808fe254ec4b1f818ed3198d5aea3d
W20260923 17:31:13.391100 17244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:13.392391 17244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:13.393034 17244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:13.394279 16826 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.392007 17340 raft_consensus.cc:2258] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:13.396435 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.396564 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:13.397234 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ca6aa0faa244799949fd44c5db72ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.397818 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.399428 17143 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:13.401777 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.402823 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.398556 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.382484 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a629ce6ab14d12b6a83ce6bc56f70f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:13.406649 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.407078 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:13.407569 17039 raft_consensus.cc:3037] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.405929 17013 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:13.409523 17041 raft_consensus.cc:2454] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:13.412350 17314 raft_consensus.cc:482] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260923 17:31:13.412854 16678 consensus_peers.cc:487] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21fc7428f55041b6b939a8ca08574b90. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:13.412786 17314 raft_consensus.cc:504] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.414618 17314 leader_election.cc:288] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:13.416507 17037 raft_consensus.cc:2379] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:13.417795 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:13.419080 16972 leader_election.cc:341] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.420486 16678 leader_election.cc:302] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:13.421521 17294 raft_consensus.cc:3037] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.428999 16827 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217
I20260923 17:31:13.430536 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.432039 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:13.432336 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:13.436473 16972 leader_election.cc:341] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.436884 16972 leader_election.cc:302] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:13.437816 17314 raft_consensus.cc:2726] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:13.438493 17338 log.cc:924] Time spent T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
I20260923 17:31:13.442374 17039 raft_consensus.cc:2454] T 87b3350e2ed247e5b600405800993ddd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:13.446532 17294 raft_consensus.cc:2726] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.447947 16678 leader_election.cc:302] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:13.446020 17343 tablet_replica.cc:330] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:13.450235 17343 raft_consensus.cc:2229] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:13.450809 17343 raft_consensus.cc:2258] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:13.452658 17343 ts_tablet_manager.cc:1905] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:13.453054 17294 raft_consensus.cc:2781] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:13.455020 17294 raft_consensus.cc:686] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:13.455929 17294 consensus_queue.cc:237] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.463712 16381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 27) took 1032 ms (client timeout 1000 ms). Trace:
I20260923 17:31:13.465503 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:13.466122 17163 tablet_replica.cc:330] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:13.466773 17163 raft_consensus.cc:2229] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:13.467386 17163 pending_rounds.cc:62] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:13.467566 17163 pending_rounds.cc:68] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596417357897728 op_type: NO_OP noop_request { }
I20260923 17:31:13.467855 17163 raft_consensus.cc:2866] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:13.466526 17292 consensus_queue.cc:1035] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.468120 17163 raft_consensus.cc:2258] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:13.470608 17163 ts_tablet_manager.cc:1905] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:13.468379 16381 rpcz_store.cc:269] 0923 17:31:12.431803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:12.439506 (+ 7703us) service_pool.cc:227] Handling call
0923 17:31:12.465904 (+ 26398us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:13.463633 (+997729us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:13.426905 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:13.427292 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:13.427314 (+ 22us) write_op.cc:435] Acquired schema lock
0923 17:31:13.427325 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:13.427579 (+ 254us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:13.427619 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:13.427640 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:13.427780 (+ 140us) tablet.cc:699] Row locks acquired
0923 17:31:13.427793 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:13.427896 (+ 103us) write_op.cc:270] Start()
0923 17:31:13.427974 (+ 78us) write_op.cc:276] Timestamp: P: 1790184673427861 usec, L: 0
0923 17:31:13.427988 (+ 14us) op_driver.cc:331] REPLICATION: starting
0923 17:31:13.437273 (+ 9285us) log.cc:842] Serialized 536 byte log entry
0923 17:31:13.449274 (+ 12001us) op_driver.cc:444] REPLICATION: finished
0923 17:31:13.450338 (+ 1064us) write_op.cc:301] APPLY: starting
0923 17:31:13.450407 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:13.450509 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:13.450617 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:13.454689 (+ 4072us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:13.454855 (+ 166us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:13.454906 (+ 51us) write_op.cc:312] APPLY: finished
0923 17:31:13.455143 (+ 237us) log.cc:842] Serialized 23 byte log entry
0923 17:31:13.455352 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:13.455420 (+ 68us) write_op.cc:454] Released schema lock
0923 17:31:13.455452 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:13.455665 (+ 213us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:13.455689 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":55621888,"child_traces":[["op",{"apply.queue_time_us":884,"apply.run_cpu_time_us":1690,"apply.run_wall_time_us":5523,"num_ops":1,"prepare.queue_time_us":318407,"raft.queue_time_us":136,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":1588,"replication_time_us":21175}]]}
I20260923 17:31:13.483503 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:13.485453 17144 ts_tablet_manager.cc:1428] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.110s user 0.005s sys 0.001s
I20260923 17:31:13.487121 17144 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:13.436506 17146 consensus_queue.cc:260] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.500905 17244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:31:13.501377 17244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:13.501756 17244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:31:13.502138 17244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:31:13.505297 17146 ts_tablet_manager.cc:1428] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.357s user 0.007s sys 0.000s
I20260923 17:31:13.508173 17146 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.511639 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_49b13ac015354ceaa1f917b54abe16f6" } modify_external_catalogs: true
I20260923 17:31:13.511782 16382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b43616adac0b44a497e85e95855ff229" } modify_external_catalogs: true
I20260923 17:31:13.512243 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_49b13ac015354ceaa1f917b54abe16f6" } modify_external_catalogs: true
I20260923 17:31:13.512379 16382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b43616adac0b44a497e85e95855ff229" } modify_external_catalogs: true
I20260923 17:31:13.542501 17146 tablet_bootstrap.cc:654] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.544293 17163 ts_tablet_manager.cc:1918] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:13.547572 17163 log.cc:1196] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:13.548375 17163 ts_tablet_manager.cc:1939] T 21fc7428f55041b6b939a8ca08574b90 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:13.544823 17144 tablet_bootstrap.cc:654] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.551820 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 21fc7428f55041b6b939a8ca08574b90 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
W20260923 17:31:13.442091 17338 consensus_queue.cc:1159] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c36e79694e154f84a46980719217d7f2" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260923 17:31:13.554601 17147 raft_consensus.cc:2781] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:13.555032 17147 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:13.555341 17147 raft_consensus.cc:3037] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.558581 17163 tablet_replica.cc:330] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:13.560530 17163 raft_consensus.cc:2229] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:13.561105 17163 raft_consensus.cc:2258] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:13.562158 17314 raft_consensus.cc:482] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.562390 17146 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:13.562711 17146 ts_tablet_manager.cc:1397] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.000s
I20260923 17:31:13.562572 17314 raft_consensus.cc:504] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.564355 17314 leader_election.cc:288] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:13.564719 17146 raft_consensus.cc:348] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.566085 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:13.566445 17163 ts_tablet_manager.cc:1905] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:13.568557 16972 leader_election.cc:341] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.571219 17146 raft_consensus.cc:374] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.571504 17146 raft_consensus.cc:729] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.573184 17144 tablet_bootstrap.cc:492] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:13.573613 17144 ts_tablet_manager.cc:1397] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.088s user 0.010s sys 0.000s
I20260923 17:31:13.575690 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.576098 17144 raft_consensus.cc:348] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.576797 17144 raft_consensus.cc:374] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.577035 17144 raft_consensus.cc:729] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:13.577482 16972 leader_election.cc:341] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.577919 16972 leader_election.cc:302] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:13.578945 17314 raft_consensus.cc:2726] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.581666 17343 ts_tablet_manager.cc:1918] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:13.579756 17144 consensus_queue.cc:260] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.581916 17340 ts_tablet_manager.cc:1905] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:13.590287 17144 ts_tablet_manager.cc:1428] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260923 17:31:13.591820 17244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:33707,127.15.246.189:45151,127.15.246.188:46075
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.246.188
--webserver_port=33399
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:13.594245 17244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:13.595247 16380 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_828238fcfb3446679de0ee7995c81b99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:13.596521 17244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:13.596884 16380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_828238fcfb3446679de0ee7995c81b99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.583334 17146 consensus_queue.cc:260] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.600430 17343 log.cc:1196] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/21a629ce6ab14d12b6a83ce6bc56f70f
I20260923 17:31:13.603057 17343 ts_tablet_manager.cc:1939] T 21a629ce6ab14d12b6a83ce6bc56f70f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:13.606705 17294 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.607097 17294 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.608840 17294 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.591231 17352 tablet.cc:1722] T 21cf533908d04a3a8ca087355528556f P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:13.621130 17147 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.622134 17294 consensus_queue.cc:1035] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.626470 17144 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:13.601536 17146 ts_tablet_manager.cc:1428] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260923 17:31:13.629004 17146 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.633569 17146 tablet_bootstrap.cc:654] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.616499 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:13.635914 17037 raft_consensus.cc:2454] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:13.636561 16387 master_service.cc:946] ReplaceTablet: received request to replace tablet f124169d87c240a5bd544d3b6693323f from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:13.637174 16678 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:13.639859 17294 raft_consensus.cc:2781] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:13.640174 17294 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:13.640492 17294 raft_consensus.cc:3037] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.647975 17294 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.650872 17146 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:13.651289 17146 ts_tablet_manager.cc:1397] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.011s
I20260923 17:31:13.652248 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 21a629ce6ab14d12b6a83ce6bc56f70f (table default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]) successfully deleted
I20260923 17:31:13.653373 17146 raft_consensus.cc:348] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.655207 17146 raft_consensus.cc:374] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.655427 17146 raft_consensus.cc:729] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.658419 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:13.658740 17146 consensus_queue.cc:260] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.659231 16893 raft_consensus.cc:2379] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:13.660208 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.660971 16891 raft_consensus.cc:2379] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be93fda0843b44f5abc49e6f35c69217 in current term 1: Already voted for candidate c36e79694e154f84a46980719217d7f2 in this term.
I20260923 17:31:13.663383 17144 tablet_bootstrap.cc:654] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.668017 17146 ts_tablet_manager.cc:1428] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.016s user 0.005s sys 0.002s
I20260923 17:31:13.670413 17146 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.670020 17353 tablet.cc:1722] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260923 17:31:13.674568 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 56906.6us
I20260923 17:31:13.680624 17146 tablet_bootstrap.cc:654] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:13.681186 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a4ca6aa0faa244799949fd44c5db72ed because the tablet was not found. No further retry: Not found: Tablet not found: a4ca6aa0faa244799949fd44c5db72ed
W20260923 17:31:13.682420 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c204e5fffe4f490295a12b8c93818c3c has no consensus state
W20260923 17:31:13.682621 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a182b6ced72428e9638465f0cc4e7ed already in progress: creating tablet
I20260923 17:31:13.683699 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:13.684491 16743 raft_consensus.cc:2379] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:13.684458 17343 tablet_replica.cc:330] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:13.685276 17343 raft_consensus.cc:2229] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:13.685798 17343 raft_consensus.cc:2258] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260923 17:31:13.689386 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:13.699991 17143 tablet_bootstrap.cc:654] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.693711 17147 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:13.701745 16827 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 272830us
I20260923 17:31:13.706267 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
W20260923 17:31:13.707605 16678 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.708003 16678 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:13.708408 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_0cb3da9761eb4c00b832368882e9e6cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:13.708761 17292 raft_consensus.cc:2726] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.710180 17294 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.717322 17163 ts_tablet_manager.cc:1918] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260923 17:31:13.730774 16384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fba670ca9ed0470e95dc7eb77bca73e4 has no consensus state
I20260923 17:31:13.745023 17163 log.cc:1196] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:13.745960 17163 ts_tablet_manager.cc:1939] T 29d44ee57faf4a1d99e077fd781a519c P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:13.747563 17343 ts_tablet_manager.cc:1905] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:13.754202 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
W20260923 17:31:13.756899 16827 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:31:13.757434 17144 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:13.757377 16827 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:13.758217 17147 raft_consensus.cc:2726] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.759605 17147 consensus_queue.cc:1035] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.734014 17332 consensus_queue.cc:1035] T 21cf533908d04a3a8ca087355528556f P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260923 17:31:13.762290 17144 ts_tablet_manager.cc:1397] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.139s user 0.011s sys 0.000s
I20260923 17:31:13.763926 17358 consensus_queue.cc:1035] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:13.767753 17367 raft_consensus.cc:482] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.768385 17367 raft_consensus.cc:504] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.770112 17358 raft_consensus.cc:482] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.770094 17144 raft_consensus.cc:348] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.770746 17144 raft_consensus.cc:374] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.771009 17144 raft_consensus.cc:729] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.770637 17358 raft_consensus.cc:504] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.772859 17358 leader_election.cc:288] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.773475 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff22ffe24d424b1e8659579febfe1270" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:13.773952 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_2243b815d62f461ca66f77ac0701c362"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:13.774433 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:13.775076 16972 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.775385 16744 raft_consensus.cc:2454] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:13.776747 16826 leader_election.cc:302] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:13.777628 17358 raft_consensus.cc:2781] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:13.778018 17358 raft_consensus.cc:482] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:13.778379 17358 raft_consensus.cc:3037] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.778597 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff22ffe24d424b1e8659579febfe1270" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:13.779893 16972 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.780273 16972 leader_election.cc:302] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:13.781051 17363 raft_consensus.cc:2726] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:13.782207 17367 leader_election.cc:288] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:13.783861 17144 consensus_queue.cc:260] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.784461 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:13.785063 17039 raft_consensus.cc:2454] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:13.789340 17340 ts_tablet_manager.cc:1918] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:13.789912 17146 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:13.790277 17146 ts_tablet_manager.cc:1397] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.120s user 0.013s sys 0.008s
I20260923 17:31:13.794487 17144 ts_tablet_manager.cc:1428] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260923 17:31:13.748430 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 29d44ee57faf4a1d99e077fd781a519c (table default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]) successfully deleted
W20260923 17:31:13.798070 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 49683.2us
I20260923 17:31:13.792398 17146 raft_consensus.cc:348] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.799311 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2243b815d62f461ca66f77ac0701c362 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.799535 17146 raft_consensus.cc:374] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.799772 17146 raft_consensus.cc:729] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.800989 17144 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:13.803125 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cb3da9761eb4c00b832368882e9e6cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.803440 17146 consensus_queue.cc:260] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.806010 17146 ts_tablet_manager.cc:1428] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260923 17:31:13.807739 17332 consensus_queue.cc:1035] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.810530 17146 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.819350 17294 consensus_queue.cc:1035] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.826392 17374 tablet.cc:1722] T 21cf533908d04a3a8ca087355528556f P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:13.842351 17144 tablet_bootstrap.cc:654] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:13.853015 17146 tablet_bootstrap.cc:654] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:13.856215 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 63808fe254ec4b1f818ed3198d5aea3d (table default.Table_2b330ec3646e4cecac1e58f4d2c2d79a [id=a7d087805c664f8f8c600e691a19d9f7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 63808fe254ec4b1f818ed3198d5aea3d consensus state has no leader
I20260923 17:31:13.860802 17144 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:13.863395 17144 ts_tablet_manager.cc:1397] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.068s user 0.007s sys 0.003s
I20260923 17:31:13.867848 17143 tablet_bootstrap.cc:492] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:13.868131 17144 raft_consensus.cc:348] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.868803 17144 raft_consensus.cc:374] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.869048 17144 raft_consensus.cc:729] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.874414 17146 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:13.874779 17146 ts_tablet_manager.cc:1397] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.002s
I20260923 17:31:13.886015 17143 ts_tablet_manager.cc:1397] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.487s user 0.016s sys 0.030s
I20260923 17:31:13.898454 17144 consensus_queue.cc:260] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.900599 17350 raft_consensus.cc:482] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.909838 17350 raft_consensus.cc:504] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.902446 16827 consensus_peers.cc:487] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21a629ce6ab14d12b6a83ce6bc56f70f. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:13.905054 17340 log.cc:1196] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:13.916582 16380 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2ac2dde5fd1045f49876d98fabeda9de" } new_table_name: "default.Table_fcfadb32897349d4962baa0dc45bd2da" modify_external_catalogs: true
I20260923 17:31:13.921445 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_76bda21bbdc644518bea95216cd34ea7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:13.922545 17292 raft_consensus.cc:482] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.922981 17292 raft_consensus.cc:504] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:13.923695 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76bda21bbdc644518bea95216cd34ea7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:13.925328 17292 leader_election.cc:288] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.929774 17292 consensus_queue.cc:1035] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:13.932148 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:13.932899 17040 raft_consensus.cc:2454] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:13.933962 16678 leader_election.cc:302] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:13.934917 17292 raft_consensus.cc:2781] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:13.935328 17292 raft_consensus.cc:482] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:13.935704 17292 raft_consensus.cc:3037] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:13.937516 17146 raft_consensus.cc:348] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.950347 17146 raft_consensus.cc:374] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:13.951263 17144 ts_tablet_manager.cc:1428] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.085s user 0.005s sys 0.000s
I20260923 17:31:13.951669 17146 raft_consensus.cc:729] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:13.952520 17146 consensus_queue.cc:260] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.953941 17144 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:13.965135 17367 raft_consensus.cc:482] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:13.965544 17367 raft_consensus.cc:504] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.912949 17350 leader_election.cc:288] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.967203 17367 leader_election.cc:288] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:13.950500 17292 raft_consensus.cc:504] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:13.971091 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:13.971930 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
W20260923 17:31:13.972436 16972 leader_election.cc:341] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:13.972733 17039 raft_consensus.cc:3037] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:13.976150 16827 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 74122.6us
I20260923 17:31:13.980363 17292 leader_election.cc:288] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:13.981081 17146 ts_tablet_manager.cc:1428] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.106s user 0.005s sys 0.001s
I20260923 17:31:13.986469 17146 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.994762 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6" } new_table_name: "default.Table_c191488ea4084de1a6d09e6c6e93b1ed" modify_external_catalogs: true
I20260923 17:31:13.997205 17144 tablet_bootstrap.cc:654] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.001746 17146 tablet_bootstrap.cc:654] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.003985 17039 raft_consensus.cc:2454] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:14.005383 16678 leader_election.cc:302] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:14.006361 17292 raft_consensus.cc:2781] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:14.006784 17292 raft_consensus.cc:686] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:14.007599 17292 consensus_queue.cc:237] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.016772 17144 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:14.017280 17144 ts_tablet_manager.cc:1397] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.064s user 0.010s sys 0.002s
I20260923 17:31:14.023511 17146 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.023896 17146 ts_tablet_manager.cc:1397] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.005s
I20260923 17:31:14.027344 17146 raft_consensus.cc:348] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.027957 17146 raft_consensus.cc:374] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.028182 17146 raft_consensus.cc:729] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.019675 17144 raft_consensus.cc:348] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.031435 17144 raft_consensus.cc:374] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.031694 17144 raft_consensus.cc:729] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.032882 17146 consensus_queue.cc:260] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.034238 17144 consensus_queue.cc:260] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.041999 17146 ts_tablet_manager.cc:1428] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.018s user 0.002s sys 0.004s
I20260923 17:31:14.044837 17144 ts_tablet_manager.cc:1428] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260923 17:31:14.045631 17367 raft_consensus.cc:482] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.046051 17367 raft_consensus.cc:504] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.047152 17146 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.047737 17367 leader_election.cc:288] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:14.052744 17340 ts_tablet_manager.cc:1939] T 21fc7428f55041b6b939a8ca08574b90 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:14.058368 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.058807 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:14.060462 16972 leader_election.cc:341] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.060870 16972 leader_election.cc:302] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:14.061935 16972 leader_election.cc:341] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.062889 17367 raft_consensus.cc:2726] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:14.063892 17146 tablet_bootstrap.cc:654] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.063872 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.064652 16893 raft_consensus.cc:3037] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.066391 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2fa2e2d002746c8ba2b0dcf45372aa8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:14.077327 17285 consensus_queue.cc:1035] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:14.080211 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:14.089630 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:14.090862 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f124169d87c240a5bd544d3b6693323f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:14.092314 16972 leader_election.cc:341] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.092796 16972 leader_election.cc:302] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:14.093865 17367 raft_consensus.cc:2726] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:14.096468 17146 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.096853 17146 ts_tablet_manager.cc:1397] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.050s user 0.018s sys 0.000s
I20260923 17:31:14.099046 17146 raft_consensus.cc:348] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.099733 17146 raft_consensus.cc:374] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.099972 17146 raft_consensus.cc:729] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.100839 17146 consensus_queue.cc:260] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.106218 17146 ts_tablet_manager.cc:1428] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260923 17:31:14.107026 17146 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:13.992800 17358 raft_consensus.cc:504] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.116967 17358 leader_election.cc:288] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.059651 17143 raft_consensus.cc:348] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.117616 17143 raft_consensus.cc:374] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.117887 17143 raft_consensus.cc:729] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.118727 17144 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:14.118482 17143 consensus_queue.cc:260] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.119346 16891 raft_consensus.cc:2454] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:14.122051 17143 ts_tablet_manager.cc:1428] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.209s user 0.006s sys 0.001s
I20260923 17:31:14.131397 17144 tablet_bootstrap.cc:654] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.136919 17146 tablet_bootstrap.cc:654] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.140091 17144 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:14.140523 17144 ts_tablet_manager.cc:1397] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.005s
I20260923 17:31:14.143162 17144 raft_consensus.cc:348] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.143761 17144 raft_consensus.cc:374] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.143982 17144 raft_consensus.cc:729] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.146201 17144 consensus_queue.cc:260] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.147116 17014 tablet_service.cc:1467] Processing CreateTablet for tablet fba670ca9ed0470e95dc7eb77bca73e4 (DEFAULT_TABLE table=default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.147516 16721 tablet_service.cc:1467] Processing CreateTablet for tablet fba670ca9ed0470e95dc7eb77bca73e4 (DEFAULT_TABLE table=default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.148007 17144 ts_tablet_manager.cc:1428] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260923 17:31:14.148679 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba670ca9ed0470e95dc7eb77bca73e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.148115 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba670ca9ed0470e95dc7eb77bca73e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.149338 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e (DEFAULT_TABLE table=default.Table_8696ac8b31fa47da987d4daf0e56100f [id=cbbd726eeaa147238c88b50945a40a92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.150719 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.153618 17018 tablet_service.cc:1467] Processing CreateTablet for tablet d20c9b997a024151a810ea015c20e18f (DEFAULT_TABLE table=default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6 [id=ee6fa9253d554437bb19b8eae678f66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.154572 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20c9b997a024151a810ea015c20e18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.158644 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (DEFAULT_TABLE table=default.Table_dbbb33fb20264319a1ace8d6d1553144 [id=40ff0e1afb354d248c9307efd7eaabdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.159737 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbfc01e66404a5da4f30c5eff8e7f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.160899 16723 tablet_service.cc:1467] Processing CreateTablet for tablet d20c9b997a024151a810ea015c20e18f (DEFAULT_TABLE table=default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6 [id=ee6fa9253d554437bb19b8eae678f66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.161863 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20c9b997a024151a810ea015c20e18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.154590 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (DEFAULT_TABLE table=default.Table_dbbb33fb20264319a1ace8d6d1553144 [id=40ff0e1afb354d248c9307efd7eaabdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.166699 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbfc01e66404a5da4f30c5eff8e7f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.157473 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e (DEFAULT_TABLE table=default.Table_8696ac8b31fa47da987d4daf0e56100f [id=cbbd726eeaa147238c88b50945a40a92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.178750 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.180327 17143 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:14.180354 17146 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.180743 17146 ts_tablet_manager.cc:1397] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.074s user 0.009s sys 0.000s
I20260923 17:31:14.189219 17144 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:14.192545 17285 raft_consensus.cc:482] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.193017 17285 raft_consensus.cc:504] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.193696 17146 raft_consensus.cc:348] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.194406 17146 raft_consensus.cc:374] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.194708 17146 raft_consensus.cc:729] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.196401 17285 leader_election.cc:288] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.196683 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:14.197377 17041 raft_consensus.cc:2454] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:14.198531 16678 leader_election.cc:302] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:14.199373 17285 raft_consensus.cc:2781] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:14.199718 17285 raft_consensus.cc:482] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:14.200053 17285 raft_consensus.cc:3037] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.201179 17146 consensus_queue.cc:260] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.207644 17146 ts_tablet_manager.cc:1428] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.027s user 0.004s sys 0.002s
I20260923 17:31:14.216482 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 2c842eff86634bf4bc70c426b5226d66 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.218230 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c842eff86634bf4bc70c426b5226d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.219164 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 2c842eff86634bf4bc70c426b5226d66 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.220273 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c842eff86634bf4bc70c426b5226d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.220477 17343 ts_tablet_manager.cc:1918] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.220829 17343 log.cc:1196] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/1483835b2bc64873a3570ad17a079713
I20260923 17:31:14.221494 17343 ts_tablet_manager.cc:1939] T 1483835b2bc64873a3570ad17a079713 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.226207 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 105455d69eb64adc8513b1bfa97e6330 (DEFAULT_TABLE table=default.Table_3d26a0116b2b4710a7482f575d4fd172 [id=f279a3956b6446af85a44fbba1886aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.227293 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105455d69eb64adc8513b1bfa97e6330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.229054 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 105455d69eb64adc8513b1bfa97e6330 (DEFAULT_TABLE table=default.Table_3d26a0116b2b4710a7482f575d4fd172 [id=f279a3956b6446af85a44fbba1886aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.230384 17143 tablet_bootstrap.cc:654] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.232439 17146 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.235289 17144 tablet_bootstrap.cc:654] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.206888 17285 raft_consensus.cc:504] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.242491 17285 leader_election.cc:288] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.250962 16720 tablet_service.cc:1467] Processing CreateTablet for tablet a3ee9185751d49e89e1c5c85ddab1661 (DEFAULT_TABLE table=default.Table_afa52f4e296d4510a0ea5644aa44f26c [id=ce59977da5fb433e88dcfcf4e6218e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.252102 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ee9185751d49e89e1c5c85ddab1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.254014 16722 tablet_service.cc:1467] Processing CreateTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.255290 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80034cbf6354e1bbe0a0220e43251c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.256414 17146 tablet_bootstrap.cc:654] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.257313 17018 tablet_service.cc:1467] Processing CreateTablet for tablet a3ee9185751d49e89e1c5c85ddab1661 (DEFAULT_TABLE table=default.Table_afa52f4e296d4510a0ea5644aa44f26c [id=ce59977da5fb433e88dcfcf4e6218e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.272027 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 1483835b2bc64873a3570ad17a079713 (table default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]) successfully deleted
I20260923 17:31:14.272526 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:14.272867 17285 raft_consensus.cc:482] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.273370 17041 raft_consensus.cc:3037] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.273407 17285 raft_consensus.cc:504] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.281692 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105455d69eb64adc8513b1bfa97e6330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.284605 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ee9185751d49e89e1c5c85ddab1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.281759 17146 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.287955 17146 ts_tablet_manager.cc:1397] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.005s
I20260923 17:31:14.290647 17146 raft_consensus.cc:348] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.291260 17146 raft_consensus.cc:374] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.291505 17146 raft_consensus.cc:729] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.279982 17041 raft_consensus.cc:2454] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:14.295445 17146 consensus_queue.cc:260] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.297526 17367 raft_consensus.cc:482] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.297927 17367 raft_consensus.cc:504] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.299660 17367 leader_election.cc:288] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:14.293217 16678 leader_election.cc:302] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:14.300424 17294 raft_consensus.cc:2781] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:14.300882 17294 raft_consensus.cc:686] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:14.301534 17294 consensus_queue.cc:237] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.303194 17146 ts_tablet_manager.cc:1428] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20260923 17:31:14.304028 17146 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.304958 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:14.306989 16972 leader_election.cc:341] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.315352 17343 tablet_replica.cc:330] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.319334 17343 raft_consensus.cc:2229] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.322322 17285 leader_election.cc:288] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.324651 17343 raft_consensus.cc:2258] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.325284 17014 tablet_service.cc:1467] Processing CreateTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.326314 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80034cbf6354e1bbe0a0220e43251c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.323041 17146 tablet_bootstrap.cc:654] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.328356 16716 tablet_service.cc:1467] Processing CreateTablet for tablet c204e5fffe4f490295a12b8c93818c3c (DEFAULT_TABLE table=default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.329468 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c204e5fffe4f490295a12b8c93818c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.321430 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:14.344442 17041 raft_consensus.cc:2454] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:14.345963 16678 leader_election.cc:302] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:14.347071 17285 raft_consensus.cc:2781] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:14.347467 17285 raft_consensus.cc:482] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:14.347843 17285 raft_consensus.cc:3037] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.351090 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_a76ca37973d44e81a5631b2e7dc25739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:14.354192 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a76ca37973d44e81a5631b2e7dc25739 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:14.361255 17343 ts_tablet_manager.cc:1905] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.367002 17013 tablet_service.cc:1467] Processing CreateTablet for tablet c204e5fffe4f490295a12b8c93818c3c (DEFAULT_TABLE table=default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.367436 17146 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.367775 17146 ts_tablet_manager.cc:1397] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.001s
W20260923 17:31:14.370831 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d20c9b997a024151a810ea015c20e18f consensus state has no leader
I20260923 17:31:14.373128 17146 raft_consensus.cc:348] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.372148 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ed9555c29504496aac9478914859831b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:14.373780 17146 raft_consensus.cc:374] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.374017 17146 raft_consensus.cc:729] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:14.374857 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed9555c29504496aac9478914859831b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:14.374552 17146 consensus_queue.cc:260] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.377933 17146 ts_tablet_manager.cc:1428] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.003s sys 0.003s
I20260923 17:31:14.394135 17285 raft_consensus.cc:504] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.397931 17285 leader_election.cc:288] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.425266 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:14.426458 17285 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.426900 17285 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.428800 17285 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.431232 16382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet ceb787c0863847289400e62718d218a1
I20260923 17:31:14.438184 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb787c0863847289400e62718d218a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:14.441262 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:14.445605 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet f124169d87c240a5bd544d3b6693323f
I20260923 17:31:14.447077 16387 catalog_manager.cc:6565] ReplaceTablet: tablet f124169d87c240a5bd544d3b6693323f deleted and replaced by tablet e92cfe501505458ea46da095a8c0d30a
I20260923 17:31:14.102552 16826 leader_election.cc:302] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:14.449492 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 347332us
I20260923 17:31:14.453131 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:14.454624 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:14.455245 16744 raft_consensus.cc:3037] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.458568 16385 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:14.464709 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 21fc7428f55041b6b939a8ca08574b90 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
W20260923 17:31:14.465476 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a182b6ced72428e9638465f0cc4e7ed already in progress: creating tablet
W20260923 17:31:14.466251 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff22ffe24d424b1e8659579febfe1270 already in progress: creating tablet
W20260923 17:31:14.466858 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 389ecb50e0c34b4d92b5296bb17f4ab1 already in progress: creating tablet
I20260923 17:31:14.468636 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:14.469410 16744 raft_consensus.cc:2454] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:14.474406 16826 leader_election.cc:302] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:14.477111 16722 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:14.492645 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_6691348cb752455e82da3fd750e9a52b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:14.494415 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6691348cb752455e82da3fd750e9a52b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:14.497172 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_6516b958fabc4cb998751ecc4d0fbfff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:14.499063 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6516b958fabc4cb998751ecc4d0fbfff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:14.521505 16862 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:14.523703 16863 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb787c0863847289400e62718d218a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:14.526069 16864 tablet_service.cc:1467] Processing CreateTablet for tablet c204e5fffe4f490295a12b8c93818c3c (DEFAULT_TABLE table=default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.526926 16869 tablet_service.cc:1467] Processing CreateTablet for tablet a3ee9185751d49e89e1c5c85ddab1661 (DEFAULT_TABLE table=default.Table_afa52f4e296d4510a0ea5644aa44f26c [id=ce59977da5fb433e88dcfcf4e6218e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.527261 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 2c842eff86634bf4bc70c426b5226d66 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.527817 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c204e5fffe4f490295a12b8c93818c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.528915 16865 tablet_service.cc:1467] Processing CreateTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.529407 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (DEFAULT_TABLE table=default.Table_dbbb33fb20264319a1ace8d6d1553144 [id=40ff0e1afb354d248c9307efd7eaabdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.522361 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e (DEFAULT_TABLE table=default.Table_8696ac8b31fa47da987d4daf0e56100f [id=cbbd726eeaa147238c88b50945a40a92]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:14.535346 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 45336.5us
I20260923 17:31:14.538466 17144 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:14.538950 17144 ts_tablet_manager.cc:1397] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.353s user 0.008s sys 0.025s
I20260923 17:31:14.541394 17144 raft_consensus.cc:348] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.542090 17144 raft_consensus.cc:374] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.542358 17144 raft_consensus.cc:729] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.542965 17144 consensus_queue.cc:260] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.557179 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb787c0863847289400e62718d218a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:14.564380 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:14.564992 17041 raft_consensus.cc:3037] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.568364 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:14.568982 17039 raft_consensus.cc:2454] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:14.569134 17403 tablet_replica.cc:330] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:14.569842 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97e944f4e0c48db813e3bee61102c53" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:14.569968 17403 raft_consensus.cc:2229] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:14.570382 17040 raft_consensus.cc:3037] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.570323 16678 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:14.570794 17403 raft_consensus.cc:2258] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.571213 17285 raft_consensus.cc:2781] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:14.571540 17285 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:14.571825 17285 raft_consensus.cc:3037] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.573594 17144 ts_tablet_manager.cc:1428] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.034s user 0.003s sys 0.002s
I20260923 17:31:14.574519 17144 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:14.574602 17343 ts_tablet_manager.cc:1918] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.574988 17343 log.cc:1196] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:14.575637 17343 ts_tablet_manager.cc:1939] T 389ecb50e0c34b4d92b5296bb17f4ab1 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.579286 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c204e5fffe4f490295a12b8c93818c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.580505 17146 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.581037 17292 consensus_queue.cc:1035] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:14.585523 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_bdb104858f024c16a05173856e5c422e" } new_table_name: "default.Table_206108d7a4384c5e8a3201f96c3cb705" modify_external_catalogs: true
I20260923 17:31:14.596206 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 0721fa178c0347598351b9d3512089b8
I20260923 17:31:14.598531 16386 catalog_manager.cc:6565] ReplaceTablet: tablet 0721fa178c0347598351b9d3512089b8 deleted and replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6
I20260923 17:31:14.598893 16386 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 118) took 1254 ms. Trace:
I20260923 17:31:14.599171 16386 rpcz_store.cc:276] 0923 17:31:13.344019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.344144 (+ 125us) service_pool.cc:227] Handling call
0923 17:31:14.598838 (+1254694us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:14.492808 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:14.493394 (+ 586us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:14.493419 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:14.493432 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:14.493887 (+ 455us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:14.493923 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:14.493943 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:14.494236 (+ 293us) tablet.cc:699] Row locks acquired
0923 17:31:14.494251 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:14.494378 (+ 127us) write_op.cc:270] Start()
0923 17:31:14.494469 (+ 91us) write_op.cc:276] Timestamp: P: 1790184674494330 usec, L: 0
0923 17:31:14.494484 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:14.495329 (+ 845us) log.cc:842] Serialized 665 byte log entry
0923 17:31:14.558224 (+ 62895us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b58000a01b0. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127589834 00007f912758e771 00007f912758e6f9 00007f912758e689 00007f912758e651 00007f912758d81d 0000000000580784 00000000005805b9 00007f91206ba6b6 00007f91206bdb11
0923 17:31:14.586865 (+ 28641us) op_driver.cc:444] REPLICATION: finished
0923 17:31:14.587336 (+ 471us) write_op.cc:301] APPLY: starting
0923 17:31:14.587416 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:14.587563 (+ 147us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:14.587592 (+ 29us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:14.587997 (+ 405us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:14.588248 (+ 251us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:14.588298 (+ 50us) write_op.cc:312] APPLY: finished
0923 17:31:14.588534 (+ 236us) log.cc:842] Serialized 29 byte log entry
0923 17:31:14.588699 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.588776 (+ 77us) write_op.cc:454] Released schema lock
0923 17:31:14.588811 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:14.589019 (+ 208us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.589056 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":278,"apply.run_cpu_time_us":1951,"apply.run_wall_time_us":1952,"num_ops":2,"prepare.queue_time_us":29567,"prepare.run_cpu_time_us":3278,"prepare.run_wall_time_us":4501,"raft.queue_time_us":623,"raft.run_cpu_time_us":1220,"raft.run_wall_time_us":62115,"replication_time_us":92313,"spinlock_wait_cycles":142406272,"thread_start_us":1642,"threads_started":1}]]}
I20260923 17:31:14.614710 17144 tablet_bootstrap.cc:654] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.616806 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:14.619858 16871 tablet_service.cc:1467] Processing CreateTablet for tablet d20c9b997a024151a810ea015c20e18f (DEFAULT_TABLE table=default.Table_1c7bb56e37544fcd9adb8e3e0856e9b6 [id=ee6fa9253d554437bb19b8eae678f66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.625061 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 105455d69eb64adc8513b1bfa97e6330 (DEFAULT_TABLE table=default.Table_3d26a0116b2b4710a7482f575d4fd172 [id=f279a3956b6446af85a44fbba1886aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.627148 17332 raft_consensus.cc:2781] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:14.627723 17332 raft_consensus.cc:686] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:14.631136 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:14.634377 17146 tablet_bootstrap.cc:654] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.637517 17147 raft_consensus.cc:482] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.638042 17147 raft_consensus.cc:504] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.639767 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:14.641731 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 73) took 1246 ms. Trace:
I20260923 17:31:14.642095 17343 rpcz_store.cc:276] 0923 17:31:13.395427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.400052 (+ 4625us) service_pool.cc:227] Handling call
0923 17:31:14.315171 (+915119us) ts_tablet_manager.cc:1145] Deleting tablet 389ecb50e0c34b4d92b5296bb17f4ab1
0923 17:31:14.315224 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:14.451043 (+135819us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.574334 (+123291us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.641664 (+ 67330us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":911969}
I20260923 17:31:14.644480 17343 tablet_replica.cc:330] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.645678 17343 raft_consensus.cc:2229] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.646382 17343 raft_consensus.cc:2258] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.646317 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (table default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]) successfully deleted
I20260923 17:31:14.647583 16889 raft_consensus.cc:3037] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:14.633881 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80034cbf6354e1bbe0a0220e43251c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.649240 17343 ts_tablet_manager.cc:1905] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.650815 17041 raft_consensus.cc:2454] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:14.653659 17040 raft_consensus.cc:2454] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:14.657243 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:14.658309 16873 tablet_service.cc:1467] Processing CreateTablet for tablet fba670ca9ed0470e95dc7eb77bca73e4 (DEFAULT_TABLE table=default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:14.660535 16386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_e69cd0d31b464f32a2c86b830f18b7df" } modify_external_catalogs: true
I20260923 17:31:14.666014 16386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_e69cd0d31b464f32a2c86b830f18b7df" } modify_external_catalogs: true
I20260923 17:31:14.669638 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ee9185751d49e89e1c5c85ddab1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.670955 17367 raft_consensus.cc:482] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.671439 17367 raft_consensus.cc:504] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.672578 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbfc01e66404a5da4f30c5eff8e7f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.674100 17367 leader_election.cc:288] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:14.676445 16384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_6516b958fabc4cb998751ecc4d0fbfff" } modify_external_catalogs: true
I20260923 17:31:14.676995 16384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_6516b958fabc4cb998751ecc4d0fbfff" } modify_external_catalogs: true
I20260923 17:31:14.688514 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.690886 17146 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.691363 17146 ts_tablet_manager.cc:1397] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.002s
I20260923 17:31:14.649178 17147 leader_election.cc:288] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.697702 17146 raft_consensus.cc:348] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.698403 17146 raft_consensus.cc:374] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.698640 17146 raft_consensus.cc:729] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.691208 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.702018 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceb787c0863847289400e62718d218a1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:14.699204 17146 consensus_queue.cc:260] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.708629 17146 ts_tablet_manager.cc:1428] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.017s user 0.007s sys 0.001s
I20260923 17:31:14.707963 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbb9840f8b6d464caae440fb03a7be54" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.711944 17340 tablet_replica.cc:330] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:14.712921 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba670ca9ed0470e95dc7eb77bca73e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.715912 17146 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.717990 17332 consensus_queue.cc:237] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.726203 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:14.733100 17343 ts_tablet_manager.cc:1918] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:14.733749 17146 tablet_bootstrap.cc:654] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.736264 17343 log.cc:1196] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/8879dee684214c6dac3f9d6642a6d97d
I20260923 17:31:14.740712 17367 raft_consensus.cc:482] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.741132 17367 raft_consensus.cc:504] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.743029 17367 leader_election.cc:288] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:14.749063 17343 ts_tablet_manager.cc:1939] T 8879dee684214c6dac3f9d6642a6d97d P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.752887 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 68) took 1397 ms. Trace:
I20260923 17:31:14.753240 17343 rpcz_store.cc:276] 0923 17:31:13.355684 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.358889 (+ 3205us) service_pool.cc:227] Handling call
0923 17:31:14.644293 (+1285404us) ts_tablet_manager.cc:1145] Deleting tablet 8879dee684214c6dac3f9d6642a6d97d
0923 17:31:14.644348 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:14.705698 (+ 61350us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.732843 (+ 27145us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.752816 (+ 19973us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1238802}
I20260923 17:31:14.754761 17343 tablet_replica.cc:330] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.758023 17343 raft_consensus.cc:2229] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.758517 17343 raft_consensus.cc:2258] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.762053 17146 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.762490 17146 ts_tablet_manager.cc:1397] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.002s
I20260923 17:31:14.764760 17146 raft_consensus.cc:348] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.765416 17146 raft_consensus.cc:374] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.765514 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db3bfd8b67a4290a007de5b4b9d0e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.765641 17146 raft_consensus.cc:729] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:14.766384 17146 consensus_queue.cc:260] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.760457 17343 ts_tablet_manager.cc:1905] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.772583 17146 ts_tablet_manager.cc:1428] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260923 17:31:14.778153 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105455d69eb64adc8513b1bfa97e6330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.781181 17146 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:14.784513 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c842eff86634bf4bc70c426b5226d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.786660 17146 tablet_bootstrap.cc:654] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:14.787066 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20c9b997a024151a810ea015c20e18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:14.796639 17343 ts_tablet_manager.cc:1918] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.797771 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.798964 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2beb8360d54efeb2fa74c005d011d1" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.799903 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:14.797046 17343 log.cc:1196] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/ff22ffe24d424b1e8659579febfe1270
W20260923 17:31:14.801174 16972 leader_election.cc:341] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.801802 16972 leader_election.cc:302] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:14.802246 17343 ts_tablet_manager.cc:1939] T ff22ffe24d424b1e8659579febfe1270 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.803345 17367 raft_consensus.cc:2726] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:14.804443 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 75) took 1408 ms. Trace:
I20260923 17:31:14.804742 17343 rpcz_store.cc:276] 0923 17:31:13.395974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.400083 (+ 4109us) service_pool.cc:227] Handling call
0923 17:31:14.754592 (+1354509us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:14.754634 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:14.777576 (+ 22942us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.796386 (+ 18810us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.804381 (+ 7995us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1347573}
I20260923 17:31:14.810577 17343 tablet_replica.cc:330] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.812335 17343 raft_consensus.cc:2229] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.815482 17343 raft_consensus.cc:2258] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.818692 17343 ts_tablet_manager.cc:1905] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:14.835335 16972 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:14.836263 16972 leader_election.cc:341] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.841846 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:14.846858 16827 consensus_peers.cc:487] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:14.853703 17143 tablet_bootstrap.cc:492] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:14.854486 17143 ts_tablet_manager.cc:1397] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.674s user 0.009s sys 0.010s
I20260923 17:31:14.855362 17343 ts_tablet_manager.cc:1918] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.855782 17343 log.cc:1196] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:14.856480 17343 ts_tablet_manager.cc:1939] T 8d65cfb2eb3648faab5869bba69cab3f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.858566 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 72) took 1463 ms. Trace:
I20260923 17:31:14.858894 17343 rpcz_store.cc:276] 0923 17:31:13.395143 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.398224 (+ 3081us) service_pool.cc:227] Handling call
0923 17:31:14.810381 (+1412157us) ts_tablet_manager.cc:1145] Deleting tablet 8d65cfb2eb3648faab5869bba69cab3f
0923 17:31:14.810433 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:14.838460 (+ 28027us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.855099 (+ 16639us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:14.858507 (+ 3408us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1401101}
I20260923 17:31:14.862409 17367 raft_consensus.cc:482] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.862818 17367 raft_consensus.cc:504] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.864727 17367 leader_election.cc:288] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:14.868783 17343 tablet_replica.cc:330] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
W20260923 17:31:14.868999 16972 leader_election.cc:341] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Not found: Tablet not found: fba670ca9ed0470e95dc7eb77bca73e4
I20260923 17:31:14.870373 17343 raft_consensus.cc:2229] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.870814 17343 raft_consensus.cc:2258] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.872668 17343 ts_tablet_manager.cc:1905] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.901892 17405 raft_consensus.cc:2726] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:14.903200 17343 ts_tablet_manager.cc:1918] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.903535 17343 log.cc:1196] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/ceb787c0863847289400e62718d218a1
I20260923 17:31:14.910784 17340 raft_consensus.cc:2229] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.904109 17343 ts_tablet_manager.cc:1939] T ceb787c0863847289400e62718d218a1 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.912770 17340 raft_consensus.cc:2258] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.915459 17340 ts_tablet_manager.cc:1905] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.917768 17343 tablet_replica.cc:330] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.918649 17343 raft_consensus.cc:2229] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.919098 17343 raft_consensus.cc:2258] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.920753 17343 ts_tablet_manager.cc:1905] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.929155 17367 raft_consensus.cc:482] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.929636 17367 raft_consensus.cc:504] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.930796 17143 raft_consensus.cc:348] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.931514 17367 leader_election.cc:288] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:14.931563 17143 raft_consensus.cc:374] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.931823 17143 raft_consensus.cc:729] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:14.937641 16972 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Not found: Tablet not found: d20c9b997a024151a810ea015c20e18f
I20260923 17:31:14.616150 17285 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.950935 17403 ts_tablet_manager.cc:1905] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:14.951429 17285 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:14.953821 16742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:14.961628 16972 leader_election.cc:341] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.962062 16972 leader_election.cc:302] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:14.963047 17367 raft_consensus.cc:2726] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:14.962984 17143 consensus_queue.cc:260] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.965559 17343 ts_tablet_manager.cc:1918] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:14.965935 17343 log.cc:1196] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:14.966549 17343 ts_tablet_manager.cc:1939] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:14.966653 17143 ts_tablet_manager.cc:1428] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.112s user 0.005s sys 0.001s
I20260923 17:31:14.968044 17343 tablet_replica.cc:330] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:14.968801 17343 raft_consensus.cc:2229] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:14.969108 17143 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:14.969274 17343 raft_consensus.cc:2258] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:14.976549 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2beb8360d54efeb2fa74c005d011d1" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:14.977746 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:14.980556 16972 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:14.981079 17146 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:14.980998 16972 leader_election.cc:302] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:14.981489 17146 ts_tablet_manager.cc:1397] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.201s user 0.034s sys 0.052s
W20260923 17:31:14.981557 16972 raft_consensus.cc:2670] T eb2beb8360d54efeb2fa74c005d011d1 P e5c071cdf65042a59cad226a15a5532c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:14.982441 17340 ts_tablet_manager.cc:1918] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:14.982831 17340 log.cc:1196] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/f124169d87c240a5bd544d3b6693323f
I20260923 17:31:14.983637 17340 ts_tablet_manager.cc:1939] T f124169d87c240a5bd544d3b6693323f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:14.983852 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:14.983757 17146 raft_consensus.cc:348] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.984445 17146 raft_consensus.cc:374] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:14.984676 17146 raft_consensus.cc:729] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:14.987679 16677 leader_election.cc:341] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:14.991231 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e2fa2e2d002746c8ba2b0dcf45372aa8 consensus state has no leader
I20260923 17:31:14.994513 17292 raft_consensus.cc:482] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:14.995004 17292 raft_consensus.cc:504] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:14.996891 17292 leader_election.cc:288] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:14.998319 16718 consensus_meta.cc:287] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.660s user 0.009s sys 0.000s
I20260923 17:31:15.000520 16678 leader_election.cc:302] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.003845 17294 raft_consensus.cc:2781] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:15.004388 17294 raft_consensus.cc:686] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
W20260923 17:31:15.004650 16972 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.005108 16972 leader_election.cc:302] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.005244 17294 consensus_queue.cc:237] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.008601 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b3350e2ed247e5b600405800993ddd" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:15.008764 16677 leader_election.cc:341] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.010376 17367 raft_consensus.cc:2726] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:15.010896 16972 leader_election.cc:341] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.012179 16972 leader_election.cc:302] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.013270 17367 raft_consensus.cc:2726] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.014444 17367 raft_consensus.cc:482] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.014855 17367 raft_consensus.cc:504] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.016731 17367 leader_election.cc:288] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:15.022145 17143 tablet_bootstrap.cc:654] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.025944 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:15.023151 16677 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:15.030434 16677 leader_election.cc:341] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:15.031468 16677 leader_election.cc:341] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.032100 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b33b5b1e1e084f889366e77bd5fa0871"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:15.034534 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b33b5b1e1e084f889366e77bd5fa0871 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:15.037755 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 8879dee684214c6dac3f9d6642a6d97d (table default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]) successfully deleted
I20260923 17:31:15.038532 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet ff22ffe24d424b1e8659579febfe1270 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
I20260923 17:31:15.039095 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 8d65cfb2eb3648faab5869bba69cab3f (table default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]) successfully deleted
I20260923 17:31:15.039641 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet ceb787c0863847289400e62718d218a1 (table default.Table_b43616adac0b44a497e85e95855ff229 [id=f3dfb81ee57147d79f9c67dbbaf451b1]) successfully deleted
I20260923 17:31:15.040926 17367 raft_consensus.cc:482] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.041419 17367 raft_consensus.cc:504] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.043444 17367 leader_election.cc:288] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:15.044657 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:15.054134 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:15.044694 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:15.060865 16972 leader_election.cc:341] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:15.061842 16972 leader_election.cc:341] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:15.062520 16972 leader_election.cc:341] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.063165 16972 leader_election.cc:302] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.065332 17367 raft_consensus.cc:2726] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.069043 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:15.079977 17143 tablet_bootstrap.cc:492] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.080428 17143 ts_tablet_manager.cc:1397] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.003s
I20260923 17:31:15.082752 17143 raft_consensus.cc:348] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.088812 16384 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 927f171ac52848248c84f804cea9a734
I20260923 17:31:15.090117 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_4b6521ef83e24b43a74029fd0c6e7326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:15.092119 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b6521ef83e24b43a74029fd0c6e7326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:15.040208 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet eb2beb8360d54efeb2fa74c005d011d1 (table default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]) successfully deleted
W20260923 17:31:15.098677 16373 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 58347.6us
I20260923 17:31:15.101045 17144 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:15.101735 17144 ts_tablet_manager.cc:1397] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.527s user 0.016s sys 0.005s
I20260923 17:31:15.102443 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_242628c220d447349a2bd325fe77290b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:15.103914 17144 raft_consensus.cc:348] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.104717 17144 raft_consensus.cc:374] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.105086 17144 raft_consensus.cc:729] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.105850 17144 consensus_queue.cc:260] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:15.110344 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 47844.9us
W20260923 17:31:15.111281 16972 leader_election.cc:341] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.111775 16972 leader_election.cc:302] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.112035 17146 consensus_queue.cc:260] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.112700 17367 raft_consensus.cc:2726] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:15.116783 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 68460.6us
I20260923 17:31:15.121485 17340 tablet_replica.cc:330] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:15.121598 17343 ts_tablet_manager.cc:1905] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:15.122851 17340 raft_consensus.cc:2229] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260923 17:31:15.128443 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 127545us
W20260923 17:31:15.135565 16677 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.138525 17146 ts_tablet_manager.cc:1428] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.157s user 0.006s sys 0.000s
I20260923 17:31:15.143736 17433 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260923 17:31:15.165689 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_242628c220d447349a2bd325fe77290b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:15.135700 17432 consensus_queue.cc:1035] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:15.137228 17144 ts_tablet_manager.cc:1428] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260923 17:31:15.167722 17144 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:15.125736 16827 consensus_peers.cc:487] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8879dee684214c6dac3f9d6642a6d97d. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:15.137538 17143 raft_consensus.cc:374] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.148653 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:15.168434 17143 raft_consensus.cc:729] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:15.150663 16826 consensus_peers.cc:487] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:15.169237 16826 leader_election.cc:302] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:15.169056 17143 consensus_queue.cc:260] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.169076 17340 raft_consensus.cc:2258] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:15.170899 17143 ts_tablet_manager.cc:1428] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.090s user 0.003s sys 0.000s
I20260923 17:31:15.171520 17332 raft_consensus.cc:2726] T 87b3350e2ed247e5b600405800993ddd P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.171911 17143 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.175102 17340 ts_tablet_manager.cc:1905] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:15.181869 17146 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:15.182567 17143 tablet_bootstrap.cc:654] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.167366 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet f124169d87c240a5bd544d3b6693323f (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]) successfully deleted
I20260923 17:31:15.193341 17435 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:15.206363 17143 tablet_bootstrap.cc:492] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.206773 17143 ts_tablet_manager.cc:1397] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.002s
I20260923 17:31:15.228658 16379 catalog_manager.cc:5495] T 87b3350e2ed247e5b600405800993ddd P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:15.230755 16379 catalog_manager.cc:5495] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:15.236600 17367 raft_consensus.cc:482] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.237121 17367 raft_consensus.cc:504] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.238996 17367 leader_election.cc:288] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:15.216315 17403 ts_tablet_manager.cc:1918] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:15.242074 17403 log.cc:1196] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/f124169d87c240a5bd544d3b6693323f
I20260923 17:31:15.242048 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:15.244290 16972 leader_election.cc:341] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.244673 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:15.246762 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:15.240408 17143 raft_consensus.cc:348] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.248534 17143 raft_consensus.cc:374] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.248797 17143 raft_consensus.cc:729] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.249815 17403 ts_tablet_manager.cc:1939] T f124169d87c240a5bd544d3b6693323f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:15.249519 17143 consensus_queue.cc:260] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.250721 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 21cf533908d04a3a8ca087355528556f (table default.Table_6bb854c7f23b41c284c4bb2b753fe3ae [id=7f9bdab909c047f09894500c06577ddd]). Expected version 2 got 0
W20260923 17:31:15.252048 16677 leader_election.cc:341] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:15.256210 16972 leader_election.cc:341] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.256832 16972 leader_election.cc:302] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.257010 17013 tablet_service.cc:1467] Processing CreateTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.258100 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a60bf118082b4e9991bfeb3ac59eb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.259049 17144 tablet_bootstrap.cc:654] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.260449 17367 raft_consensus.cc:2726] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.261943 16885 raft_consensus.cc:3037] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.275938 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]). Expected version 1 got 0
I20260923 17:31:15.277841 17143 ts_tablet_manager.cc:1428] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.071s user 0.006s sys 0.001s
I20260923 17:31:15.281282 17143 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.300722 16871 tablet_service.cc:1467] Processing CreateTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.301851 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a60bf118082b4e9991bfeb3ac59eb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.314056 16725 tablet_service.cc:1467] Processing CreateTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.315230 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a60bf118082b4e9991bfeb3ac59eb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.315716 17340 ts_tablet_manager.cc:1918] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:15.316118 17340 log.cc:1196] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/ceb787c0863847289400e62718d218a1
I20260923 17:31:15.316807 17340 ts_tablet_manager.cc:1939] T ceb787c0863847289400e62718d218a1 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:15.318189 17143 tablet_bootstrap.cc:654] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.320889 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet f124169d87c240a5bd544d3b6693323f (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]) successfully deleted
W20260923 17:31:15.326395 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb2beb8360d54efeb2fa74c005d011d1 already in progress: creating tablet
I20260923 17:31:15.332052 17427 consensus_queue.cc:1035] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260923 17:31:15.330678 17430 raft_consensus.cc:482] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.333920 17430 raft_consensus.cc:504] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.336490 17430 leader_election.cc:288] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:15.330506 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet ceb787c0863847289400e62718d218a1 (table default.Table_b43616adac0b44a497e85e95855ff229 [id=f3dfb81ee57147d79f9c67dbbaf451b1]) successfully deleted
W20260923 17:31:15.340322 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb2beb8360d54efeb2fa74c005d011d1 already in progress: creating tablet
I20260923 17:31:15.324743 16387 master_service.cc:946] ReplaceTablet: received request to replace tablet f124169d87c240a5bd544d3b6693323f from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:15.344286 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 320711us
W20260923 17:31:15.344969 16678 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:15.345309 16678 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:15.340373 17403 tablet_replica.cc:330] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:15.353144 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:15.354211 17340 tablet_replica.cc:330] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:15.355366 17285 raft_consensus.cc:2726] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.346778 17403 raft_consensus.cc:2229] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:15.356954 17340 raft_consensus.cc:2229] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:15.357486 17340 raft_consensus.cc:2258] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:15.357667 17403 pending_rounds.cc:62] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:15.357908 17403 pending_rounds.cc:68] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596425947148288 op_type: NO_OP noop_request { }
I20260923 17:31:15.359385 17340 ts_tablet_manager.cc:1905] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:15.360954 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:15.361639 17038 raft_consensus.cc:2454] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:15.362640 17403 raft_consensus.cc:2866] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:15.364960 17403 raft_consensus.cc:2258] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:15.374085 17143 tablet_bootstrap.cc:492] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.374477 17143 ts_tablet_manager.cc:1397] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.005s
I20260923 17:31:15.376889 17143 raft_consensus.cc:348] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.377549 17143 raft_consensus.cc:374] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.374202 17403 ts_tablet_manager.cc:1905] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:15.382712 16677 leader_election.cc:341] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.377781 17143 raft_consensus.cc:729] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.384866 17433 tablet.cc:1717] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Skipping requested alter to schema version 2, tablet already version 2
I20260923 17:31:15.384804 17143 consensus_queue.cc:260] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.390161 17435 tablet.cc:1717] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Skipping requested alter to schema version 2, tablet already version 2
I20260923 17:31:15.398319 17143 ts_tablet_manager.cc:1428] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260923 17:31:15.405719 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:15.405859 17143 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.415122 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:15.276095 17146 tablet_bootstrap.cc:654] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.416961 16867 tablet_service.cc:1467] Processing CreateTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (DEFAULT_TABLE table=default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.418536 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.408022 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_41fb14b41ba048a58552a94525c3436f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:15.423913 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41fb14b41ba048a58552a94525c3436f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:15.426869 17143 tablet_bootstrap.cc:654] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.434098 17340 ts_tablet_manager.cc:1918] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:15.434484 17340 log.cc:1196] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/0721fa178c0347598351b9d3512089b8
I20260923 17:31:15.435176 17340 ts_tablet_manager.cc:1939] T 0721fa178c0347598351b9d3512089b8 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:15.441884 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:15.447718 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_bdb104858f024c16a05173856e5c422e [id=004f88ba43bd4f17ace3805b35d3527d]) successfully deleted
I20260923 17:31:15.448841 17018 tablet_service.cc:1467] Processing CreateTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (DEFAULT_TABLE table=default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.449995 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.450651 17143 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.451025 17143 ts_tablet_manager.cc:1397] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.006s
I20260923 17:31:15.453109 17143 raft_consensus.cc:348] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.453740 17143 raft_consensus.cc:374] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.453984 17143 raft_consensus.cc:729] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:15.455798 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because the tablet was not found. No further retry: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:15.457420 17143 consensus_queue.cc:260] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.459714 17143 ts_tablet_manager.cc:1428] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260923 17:31:15.465945 17143 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.466883 16726 tablet_service.cc:1467] Processing CreateTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (DEFAULT_TABLE table=default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.467952 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.467082 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:15.468699 16387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ad3626a8c0e24d85a90f0629af2c5b75" } modify_external_catalogs: true
I20260923 17:31:15.469403 16387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ad3626a8c0e24d85a90f0629af2c5b75" } modify_external_catalogs: true
I20260923 17:31:15.474148 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:15.474720 17143 tablet_bootstrap.cc:654] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.475260 16388 catalog_manager.cc:5495] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:15.476567 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]). Expected version 1 got 0
I20260923 17:31:15.477497 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:15.482219 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.483184 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.489825 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.490931 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.491077 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 42ccd82f191942239ffb88459d362107 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.492066 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ccd82f191942239ffb88459d362107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.492700 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 42ccd82f191942239ffb88459d362107 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.493685 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ccd82f191942239ffb88459d362107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.495597 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 42ccd82f191942239ffb88459d362107 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.496716 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ccd82f191942239ffb88459d362107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.496941 16869 tablet_service.cc:1467] Processing CreateTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (DEFAULT_TABLE table=default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.497802 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.500923 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.501967 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.521842 17143 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.522244 17143 ts_tablet_manager.cc:1397] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.011s
I20260923 17:31:15.524264 17143 raft_consensus.cc:348] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.524925 17143 raft_consensus.cc:374] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.525162 17143 raft_consensus.cc:729] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.528501 17143 consensus_queue.cc:260] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.532697 17143 ts_tablet_manager.cc:1428] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.010s user 0.004s sys 0.002s
I20260923 17:31:15.532533 16721 tablet_service.cc:1467] Processing CreateTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (DEFAULT_TABLE table=default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.533491 17143 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.533684 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.549520 17143 tablet_bootstrap.cc:654] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.556756 17013 tablet_service.cc:1467] Processing CreateTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (DEFAULT_TABLE table=default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.557034 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 051f2e478a324df98647d90107d55533 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.558028 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.558099 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051f2e478a324df98647d90107d55533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.567535 17143 tablet_bootstrap.cc:492] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.567967 17143 ts_tablet_manager.cc:1397] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.002s
I20260923 17:31:15.570621 17143 raft_consensus.cc:348] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.571270 17143 raft_consensus.cc:374] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.571480 17143 raft_consensus.cc:729] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.572151 17143 consensus_queue.cc:260] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.576242 17143 ts_tablet_manager.cc:1428] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260923 17:31:15.577132 17143 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.583964 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b6843cebc625435d8e90cf414884277d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:15.586089 17143 tablet_bootstrap.cc:654] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:15.587003 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6843cebc625435d8e90cf414884277d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:15.598991 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 051f2e478a324df98647d90107d55533 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.599469 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:15.600162 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051f2e478a324df98647d90107d55533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.600188 17041 raft_consensus.cc:2454] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:15.604357 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 051f2e478a324df98647d90107d55533 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.605607 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051f2e478a324df98647d90107d55533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.621881 16678 leader_election.cc:302] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.623042 17285 raft_consensus.cc:2781] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:15.623383 17285 raft_consensus.cc:482] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:15.623652 17285 raft_consensus.cc:3037] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.623819 16867 tablet_service.cc:1467] Processing CreateTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 (DEFAULT_TABLE table=default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.626148 17292 consensus_queue.cc:1035] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:15.627308 17343 ts_tablet_manager.cc:1918] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:15.627942 16678 leader_election.cc:302] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:15.624716 17405 raft_consensus.cc:482] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.644318 17405 raft_consensus.cc:504] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.646378 17405 leader_election.cc:288] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:15.636332 17147 consensus_queue.cc:1035] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:15.649362 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9b734b250d4920abbafe76f52ec540" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:15.650710 16826 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.651554 17144 tablet_bootstrap.cc:492] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:15.651968 17144 ts_tablet_manager.cc:1397] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.484s user 0.025s sys 0.022s
I20260923 17:31:15.656471 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05a6f83314d433a9502e0c00ba8ad09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.651958 17343 log.cc:1196] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/f124169d87c240a5bd544d3b6693323f
I20260923 17:31:15.657676 17343 ts_tablet_manager.cc:1939] T f124169d87c240a5bd544d3b6693323f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:15.666596 16717 tablet_service.cc:1467] Processing CreateTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 (DEFAULT_TABLE table=default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.670226 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05a6f83314d433a9502e0c00ba8ad09. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:15.672699 16827 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:15.680850 16827 leader_election.cc:302] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:15.682060 17147 raft_consensus.cc:2726] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:15.690392 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 88) took 1237 ms. Trace:
I20260923 17:31:15.691648 17343 rpcz_store.cc:276] 0923 17:31:14.452774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.452924 (+ 150us) service_pool.cc:227] Handling call
0923 17:31:14.967904 (+514980us) ts_tablet_manager.cc:1145] Deleting tablet f124169d87c240a5bd544d3b6693323f
0923 17:31:14.967940 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:15.414694 (+446754us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.476007 (+ 61313us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.690330 (+214323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":514298}
I20260923 17:31:15.693509 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet f124169d87c240a5bd544d3b6693323f (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]) successfully deleted
I20260923 17:31:15.694835 17343 tablet_replica.cc:330] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:15.696199 17343 raft_consensus.cc:2229] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:15.697916 17343 raft_consensus.cc:2258] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:15.700306 17343 ts_tablet_manager.cc:1905] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:15.700402 16382 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:15.701851 16382 catalog_manager.cc:5495] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:15.704285 16382 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_2ac2dde5fd1045f49876d98fabeda9de [id=6d073df311674a358a520dd1586a3345]). Expected version 1 got 0
I20260923 17:31:15.706481 16380 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 126) took 1790 ms. Trace:
I20260923 17:31:15.706899 16380 rpcz_store.cc:276] 0923 17:31:13.916073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.916188 (+ 115us) service_pool.cc:227] Handling call
0923 17:31:13.917259 (+ 1071us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:13.918010 (+ 751us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:13.918705 (+ 695us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:13.918895 (+ 190us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:14.272077 (+353182us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:15.706263 (+1434186us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:14.150155 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:14.150520 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:14.150543 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:14.150555 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:14.150816 (+ 261us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:14.150857 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:14.150879 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:14.151029 (+ 150us) tablet.cc:699] Row locks acquired
0923 17:31:14.151041 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:14.151155 (+ 114us) write_op.cc:270] Start()
0923 17:31:14.151249 (+ 94us) write_op.cc:276] Timestamp: P: 1790184674151096 usec, L: 0
0923 17:31:14.151266 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:14.167025 (+ 15759us) log.cc:842] Serialized 405 byte log entry
0923 17:31:14.168351 (+ 1326us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:14.270276 (+101925us) op_driver.cc:444] REPLICATION: finished
0923 17:31:14.270928 (+ 652us) write_op.cc:301] APPLY: starting
0923 17:31:14.270990 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:14.271082 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:14.271105 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:14.271273 (+ 168us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:14.271397 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:14.271439 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:14.271628 (+ 189us) log.cc:842] Serialized 23 byte log entry
0923 17:31:14.271787 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.271846 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:14.271876 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:14.272400 (+ 524us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.272423 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":478,"apply.run_cpu_time_us":1314,"apply.run_wall_time_us":1651,"num_ops":1,"prepare.queue_time_us":230053,"prepare.run_cpu_time_us":3382,"prepare.run_wall_time_us":18377,"raft.queue_time_us":909,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":17,"replication_time_us":118931,"spinlock_wait_cycles":34916736}]]}
I20260923 17:31:15.712867 16380 master_service.cc:946] ReplaceTablet: received request to replace tablet 2c842eff86634bf4bc70c426b5226d66 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:15.713583 17413 consensus_queue.cc:1035] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:15.728457 17292 raft_consensus.cc:2781] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:15.728842 17292 raft_consensus.cc:482] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:15.729260 17292 raft_consensus.cc:3037] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.730139 17285 raft_consensus.cc:504] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.733472 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:15.734103 16890 raft_consensus.cc:3037] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.736963 17292 raft_consensus.cc:504] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.740350 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:15.740921 16885 raft_consensus.cc:3037] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.743594 16890 raft_consensus.cc:2454] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:15.745070 17285 leader_election.cc:288] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:15.745002 16677 leader_election.cc:302] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:15.746062 17285 raft_consensus.cc:2781] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:15.746532 17285 raft_consensus.cc:686] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:15.763907 16885 raft_consensus.cc:2454] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:15.765429 16677 leader_election.cc:302] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:15.766574 17427 raft_consensus.cc:2781] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:15.767230 17427 raft_consensus.cc:686] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
W20260923 17:31:15.768551 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 140694us
I20260923 17:31:15.770076 17010 tablet_service.cc:1467] Processing CreateTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 (DEFAULT_TABLE table=default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.771238 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05a6f83314d433a9502e0c00ba8ad09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.772141 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df53a9825e5348189485f30b8fdedc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:15.772715 17039 raft_consensus.cc:3037] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.773123 17143 tablet_bootstrap.cc:492] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.773530 17143 ts_tablet_manager.cc:1397] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.197s user 0.037s sys 0.015s
I20260923 17:31:15.781176 17039 raft_consensus.cc:2454] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:15.768014 17427 consensus_queue.cc:237] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.775992 17143 raft_consensus.cc:348] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.789608 17143 raft_consensus.cc:374] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.789911 17143 raft_consensus.cc:729] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.790481 17143 consensus_queue.cc:260] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.791673 17292 leader_election.cc:288] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:15.793143 17144 raft_consensus.cc:348] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.794008 17144 raft_consensus.cc:374] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.794268 17144 raft_consensus.cc:729] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.794790 17144 consensus_queue.cc:260] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.797297 17454 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:15.801646 17143 ts_tablet_manager.cc:1428] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.028s user 0.005s sys 0.002s
I20260923 17:31:15.801986 17433 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:15.803309 17343 ts_tablet_manager.cc:1918] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:15.825301 17343 log.cc:1196] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/0721fa178c0347598351b9d3512089b8
I20260923 17:31:15.826996 17343 ts_tablet_manager.cc:1939] T 0721fa178c0347598351b9d3512089b8 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:15.827677 17144 ts_tablet_manager.cc:1428] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.175s user 0.006s sys 0.000s
I20260923 17:31:15.828689 17144 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:15.829035 17435 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:15.829665 17343 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 89) took 1201 ms. Trace:
I20260923 17:31:15.829931 17343 rpcz_store.cc:276] 0923 17:31:14.627833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.639622 (+ 11789us) service_pool.cc:227] Handling call
0923 17:31:15.694663 (+1055041us) ts_tablet_manager.cc:1145] Deleting tablet 0721fa178c0347598351b9d3512089b8
0923 17:31:15.694704 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:15.741122 (+ 46418us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.803128 (+ 62006us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.829616 (+ 26488us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1054444}
I20260923 17:31:15.833787 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_bdb104858f024c16a05173856e5c422e [id=004f88ba43bd4f17ace3805b35d3527d]) successfully deleted
I20260923 17:31:15.840039 17403 ts_tablet_manager.cc:1918] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:15.859392 17403 log.cc:1196] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/ceb787c0863847289400e62718d218a1
I20260923 17:31:15.860802 17403 ts_tablet_manager.cc:1939] T ceb787c0863847289400e62718d218a1 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:15.869359 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 79) took 1433 ms. Trace:
I20260923 17:31:15.869803 17403 rpcz_store.cc:276] 0923 17:31:14.435771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.556981 (+121210us) service_pool.cc:227] Handling call
0923 17:31:15.332685 (+775704us) ts_tablet_manager.cc:1145] Deleting tablet ceb787c0863847289400e62718d218a1
0923 17:31:15.334611 (+ 1926us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:15.633258 (+298647us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.839765 (+206507us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:15.868716 (+ 28951us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":773829}
I20260923 17:31:15.871338 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet ceb787c0863847289400e62718d218a1 (table default.Table_b43616adac0b44a497e85e95855ff229 [id=f3dfb81ee57147d79f9c67dbbaf451b1]) successfully deleted
I20260923 17:31:15.863878 17144 tablet_bootstrap.cc:654] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.874993 17403 tablet_replica.cc:330] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:15.885311 17144 tablet_bootstrap.cc:492] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:15.885903 17144 ts_tablet_manager.cc:1397] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.002s
I20260923 17:31:15.888477 17144 raft_consensus.cc:348] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.889343 17144 raft_consensus.cc:374] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.889619 17144 raft_consensus.cc:729] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.890396 17144 consensus_queue.cc:260] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.892733 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.896931 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 191292e60d0d46fb89b52390feb48f80 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.895232 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:15.898751 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547fda6e7bcf474e9f73ba9a6659d34f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:15.905251 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because the tablet was not found. No further retry: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:15.747364 17285 consensus_queue.cc:237] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.916003 17403 raft_consensus.cc:2229] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:15.916642 17403 raft_consensus.cc:2258] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:15.918190 17143 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:15.918547 17403 ts_tablet_manager.cc:1905] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:15.921597 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547fda6e7bcf474e9f73ba9a6659d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.922137 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191292e60d0d46fb89b52390feb48f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:15.927340 17143 tablet_bootstrap.cc:654] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:15.933534 17144 ts_tablet_manager.cc:1428] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260923 17:31:15.939513 17144 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:15.942394 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8895cffac5945a19a5de7fba1319f22" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:15.943007 17036 raft_consensus.cc:3037] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.944303 17405 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.944764 17405 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.945147 17367 raft_consensus.cc:482] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:15.945578 17367 raft_consensus.cc:504] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.947698 17405 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:15.947898 17367 leader_election.cc:288] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:15.950078 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:15.950853 16743 raft_consensus.cc:2379] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:15.956832 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c842eff86634bf4bc70c426b5226d66" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:15.959298 16972 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:15.963148 17146 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:15.965376 17146 ts_tablet_manager.cc:1397] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.784s user 0.012s sys 0.005s
I20260923 17:31:15.966100 17143 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:15.966504 17143 ts_tablet_manager.cc:1397] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.006s
I20260923 17:31:15.971149 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c842eff86634bf4bc70c426b5226d66" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:15.976082 16890 raft_consensus.cc:3037] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:15.985299 17143 raft_consensus.cc:348] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:15.986063 17143 raft_consensus.cc:374] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:15.992298 17143 raft_consensus.cc:729] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:15.993170 17036 raft_consensus.cc:2454] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:15.999902 17146 raft_consensus.cc:348] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.008417 17285 raft_consensus.cc:482] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.008893 17146 raft_consensus.cc:374] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.009167 17146 raft_consensus.cc:729] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.009214 17285 raft_consensus.cc:504] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.013016 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:16.010669 17146 consensus_queue.cc:260] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.013737 16885 raft_consensus.cc:2454] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:16.015014 17143 consensus_queue.cc:260] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.015523 16677 leader_election.cc:302] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.017347 17427 raft_consensus.cc:2781] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:16.017788 17427 raft_consensus.cc:482] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:16.018157 17427 raft_consensus.cc:3037] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.022614 17429 consensus_queue.cc:1035] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:16.025164 17454 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.227s user 0.005s sys 0.000s
I20260923 17:31:16.029716 17143 ts_tablet_manager.cc:1428] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.063s user 0.006s sys 0.001s
I20260923 17:31:16.035423 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 191292e60d0d46fb89b52390feb48f80 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:16.036602 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191292e60d0d46fb89b52390feb48f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:16.037009 17143 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:16.038724 17144 tablet_bootstrap.cc:654] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.043431 17143 tablet_bootstrap.cc:654] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:16.052536 16374 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fba670ca9ed0470e95dc7eb77bca73e4 consensus state has no leader
W20260923 17:31:16.068758 16972 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.069751 16972 leader_election.cc:302] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:16.088815 16380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:16.090251 16380 catalog_manager.cc:6565] ReplaceTablet: tablet 2c842eff86634bf4bc70c426b5226d66 deleted and replaced by tablet 3208c90d96f349988acb6705f2f438d5
I20260923 17:31:16.093020 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.097929 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:16.099462 17013 tablet_service.cc:1467] Processing CreateTablet for tablet 191292e60d0d46fb89b52390feb48f80 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:16.100739 16827 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:16.101261 16827 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:16.101573 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191292e60d0d46fb89b52390feb48f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:16.101986 17405 raft_consensus.cc:3037] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.110345 17146 ts_tablet_manager.cc:1428] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.144s user 0.006s sys 0.001s
I20260923 17:31:16.118178 17146 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:16.116202 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 2 got 0
I20260923 17:31:16.109689 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:16.126603 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 2 got 0
I20260923 17:31:16.109014 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547fda6e7bcf474e9f73ba9a6659d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:16.158530 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_fcfadb32897349d4962baa0dc45bd2da [id=6d073df311674a358a520dd1586a3345]). Expected version 2 got 0
I20260923 17:31:16.159377 16386 catalog_manager.cc:5495] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:16.168769 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:16.169854 16386 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:16.172036 17343 tablet_replica.cc:330] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:16.173000 17343 raft_consensus.cc:2229] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:16.173677 17343 raft_consensus.cc:2258] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.176838 17343 ts_tablet_manager.cc:1905] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:16.187938 17367 raft_consensus.cc:482] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.188704 17367 raft_consensus.cc:504] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.191924 17367 leader_election.cc:288] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:16.220463 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:16.224396 16382 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_6bb854c7f23b41c284c4bb2b753fe3ae [id=7f9bdab909c047f09894500c06577ddd] alter complete (version 2)
W20260923 17:31:16.230291 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 2c842eff86634bf4bc70c426b5226d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c842eff86634bf4bc70c426b5226d66 already in progress: creating tablet
I20260923 17:31:16.232136 17146 tablet_bootstrap.cc:654] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.217628 16385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_bdb104858f024c16a05173856e5c422e [id=004f88ba43bd4f17ace3805b35d3527d] alter complete (version 2)
I20260923 17:31:16.233713 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:16.236485 17405 raft_consensus.cc:2726] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:16.238706 17013 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:16.245116 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.245373 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.245653 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.245842 17143 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:16.245842 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.246313 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.249225 17143 ts_tablet_manager.cc:1397] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.212s user 0.006s sys 0.031s
I20260923 17:31:16.251065 17471 tablet_replica.cc:330] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:16.251930 17471 raft_consensus.cc:2229] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
W20260923 17:31:16.236059 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 25) took 3115 ms (client timeout 1000 ms). Trace:
W20260923 17:31:16.256661 16972 leader_election.cc:341] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:16.258675 16385 rpcz_store.cc:269] 0923 17:31:13.121080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.136360 (+ 15280us) service_pool.cc:227] Handling call
0923 17:31:13.316459 (+180099us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:16.235972 (+2919513us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:14.497415 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:14.497796 (+ 381us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:14.497822 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:14.497836 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:14.498083 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:14.498117 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:14.498137 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:14.498278 (+ 141us) tablet.cc:699] Row locks acquired
0923 17:31:14.498291 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:14.498390 (+ 99us) write_op.cc:270] Start()
0923 17:31:14.498463 (+ 73us) write_op.cc:276] Timestamp: P: 1790184674498349 usec, L: 0
0923 17:31:14.498478 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:14.498895 (+ 417us) log.cc:842] Serialized 356 byte log entry
0923 17:31:14.558999 (+ 60104us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b58000a01b0. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127589834 00007f912758e771 00007f912758e6f9 00007f912758e689 00007f912758e651 00007f912758d81d 0000000000580784 00000000005805b9 00007f91206ba6b6 00007f91206bdb11
0923 17:31:14.587052 (+ 28053us) op_driver.cc:444] REPLICATION: finished
0923 17:31:14.588072 (+ 1020us) write_op.cc:301] APPLY: starting
0923 17:31:14.588122 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:14.588238 (+ 116us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:14.588257 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:14.588395 (+ 138us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:14.588517 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:14.588558 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:14.588752 (+ 194us) log.cc:842] Serialized 23 byte log entry
0923 17:31:14.588919 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.588980 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:14.589013 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:14.589223 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:14.589254 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21561,"reported_tablets":8,"spinlock_wait_cycles":408324992,"child_traces":[["op",{"apply.queue_time_us":886,"apply.run_cpu_time_us":1348,"apply.run_wall_time_us":1347,"num_ops":1,"prepare.queue_time_us":13823,"prepare.run_cpu_time_us":1719,"prepare.run_wall_time_us":1720,"raft.queue_time_us":13384,"raft.run_cpu_time_us":1630,"raft.run_wall_time_us":46563,"replication_time_us":88515,"spinlock_wait_cycles":103498496}]]}
I20260923 17:31:16.264971 17146 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:16.265488 17146 ts_tablet_manager.cc:1397] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.148s user 0.013s sys 0.007s
I20260923 17:31:16.273373 17146 raft_consensus.cc:348] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.274149 17146 raft_consensus.cc:374] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.275671 17146 raft_consensus.cc:729] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.251626 17143 raft_consensus.cc:348] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.276825 17143 raft_consensus.cc:374] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.277077 17143 raft_consensus.cc:729] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.277652 17143 consensus_queue.cc:260] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.279458 17143 ts_tablet_manager.cc:1428] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.030s user 0.002s sys 0.001s
I20260923 17:31:16.283340 17471 raft_consensus.cc:2258] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.284700 17146 consensus_queue.cc:260] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.287659 17471 ts_tablet_manager.cc:1905] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:16.291677 17146 ts_tablet_manager.cc:1428] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260923 17:31:16.292621 17146 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:16.284085 17143 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:16.299142 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db3504813e9c4698a546b6d6fa0fbcb6 has no consensus state
W20260923 17:31:16.306710 16383 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 21) took 2861 ms (client timeout 1000 ms). Trace:
W20260923 17:31:16.307123 16383 rpcz_store.cc:269] 0923 17:31:13.445605 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.445749 (+ 144us) service_pool.cc:227] Handling call
0923 17:31:16.306501 (+2860752us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3}
I20260923 17:31:16.313701 17146 tablet_bootstrap.cc:654] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:16.315104 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 29) took 1767 ms (client timeout 1000 ms). Trace:
I20260923 17:31:16.316561 16381 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 132) took 1731 ms. Trace:
I20260923 17:31:16.316653 17143 tablet_bootstrap.cc:654] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.317258 16381 rpcz_store.cc:276] 0923 17:31:14.584910 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.585067 (+ 157us) service_pool.cc:227] Handling call
0923 17:31:14.586212 (+ 1145us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:16.219551 (+1633339us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:16.240187 (+ 20636us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:16.240722 (+ 535us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:16.298393 (+ 57671us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:16.316503 (+ 18110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.255069 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.255432 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.255453 (+ 21us) write_op.cc:435] Acquired schema lock
0923 17:31:16.255465 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:16.255701 (+ 236us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.255739 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.255760 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:16.255904 (+ 144us) tablet.cc:699] Row locks acquired
0923 17:31:16.255917 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.256027 (+ 110us) write_op.cc:270] Start()
0923 17:31:16.256103 (+ 76us) write_op.cc:276] Timestamp: P: 1790184676250598 usec, L: 0
0923 17:31:16.256118 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.278410 (+ 22292us) log.cc:842] Serialized 405 byte log entry
0923 17:31:16.288054 (+ 9644us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.290228 (+ 2174us) write_op.cc:301] APPLY: starting
0923 17:31:16.290271 (+ 43us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.290353 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.290436 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.290561 (+ 125us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.290679 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.290719 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:16.290865 (+ 146us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.290964 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.291019 (+ 55us) write_op.cc:454] Released schema lock
0923 17:31:16.291043 (+ 24us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.291198 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.291222 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16719,"child_traces":[["op",{"apply.queue_time_us":1999,"apply.run_cpu_time_us":1136,"apply.run_wall_time_us":1136,"num_ops":1,"prepare.queue_time_us":13133,"raft.queue_time_us":1042,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":10,"replication_time_us":31867,"threads_started":1}]]}
W20260923 17:31:16.324498 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 26) took 1321 ms (client timeout 1000 ms). Trace:
W20260923 17:31:16.315490 16388 rpcz_store.cc:269] 0923 17:31:14.547354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.547498 (+ 144us) service_pool.cc:227] Handling call
0923 17:31:16.315020 (+1767522us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:15.594695 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:15.595118 (+ 423us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:15.595139 (+ 21us) write_op.cc:435] Acquired schema lock
0923 17:31:15.595150 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:15.595426 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:15.595465 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:15.595485 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:15.595630 (+ 145us) tablet.cc:699] Row locks acquired
0923 17:31:15.595643 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:15.595757 (+ 114us) write_op.cc:270] Start()
0923 17:31:15.595833 (+ 76us) write_op.cc:276] Timestamp: P: 1790184675595716 usec, L: 0
0923 17:31:15.595848 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:15.596353 (+ 505us) log.cc:842] Serialized 536 byte log entry
0923 17:31:15.639604 (+ 43251us) op_driver.cc:444] REPLICATION: finished
0923 17:31:15.640664 (+ 1060us) write_op.cc:301] APPLY: starting
0923 17:31:15.640721 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:15.640809 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:15.640897 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:15.641056 (+ 159us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:15.641196 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:15.641244 (+ 48us) write_op.cc:312] APPLY: finished
0923 17:31:15.641430 (+ 186us) log.cc:842] Serialized 23 byte log entry
0923 17:31:15.641573 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:15.641634 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:15.641669 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:15.641833 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:15.641855 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7012,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":874,"apply.run_cpu_time_us":1336,"apply.run_wall_time_us":1336,"num_ops":1,"prepare.queue_time_us":114229,"prepare.run_cpu_time_us":2083,"prepare.run_wall_time_us":58998,"raft.queue_time_us":36733,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":256,"replication_time_us":43673}]]}
W20260923 17:31:16.324581 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 28) took 2784 ms (client timeout 1000 ms). Trace:
I20260923 17:31:16.340920 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:16.324944 16382 rpcz_store.cc:269] 0923 17:31:15.003305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:15.005887 (+ 2582us) service_pool.cc:227] Handling call
0923 17:31:16.324427 (+1318540us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:15.894927 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:15.895263 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:15.895288 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:15.895300 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:15.895710 (+ 410us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:15.895752 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:15.895775 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:15.895915 (+ 140us) tablet.cc:699] Row locks acquired
0923 17:31:15.895927 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:15.896031 (+ 104us) write_op.cc:270] Start()
0923 17:31:15.896122 (+ 91us) write_op.cc:276] Timestamp: P: 1790184675895971 usec, L: 0
0923 17:31:15.896138 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:15.896862 (+ 724us) log.cc:842] Serialized 536 byte log entry
0923 17:31:16.062309 (+165447us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.062660 (+ 351us) write_op.cc:301] APPLY: starting
0923 17:31:16.062715 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.062801 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.062900 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.063071 (+ 171us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.063205 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.063249 (+ 44us) write_op.cc:312] APPLY: finished
0923 17:31:16.063421 (+ 172us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.063563 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.063623 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:16.063654 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.063814 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.063834 (+ 20us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:16.140589 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.140977 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.141005 (+ 28us) write_op.cc:435] Acquired schema lock
0923 17:31:16.141018 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:16.141338 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.141380 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.141406 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:16.141552 (+ 146us) tablet.cc:699] Row locks acquired
0923 17:31:16.141565 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.141692 (+ 127us) write_op.cc:270] Start()
0923 17:31:16.141782 (+ 90us) write_op.cc:276] Timestamp: P: 1790184676136265 usec, L: 0
0923 17:31:16.141799 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.142345 (+ 546us) log.cc:842] Serialized 356 byte log entry
0923 17:31:16.159847 (+ 17502us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.160238 (+ 391us) write_op.cc:301] APPLY: starting
0923 17:31:16.160292 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.160397 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.160505 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.160698 (+ 193us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.160839 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.160888 (+ 49us) write_op.cc:312] APPLY: finished
0923 17:31:16.161068 (+ 180us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.161225 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.161313 (+ 88us) write_op.cc:454] Released schema lock
0923 17:31:16.161346 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.161527 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.161552 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":217,"apply.run_cpu_time_us":1318,"apply.run_wall_time_us":1317,"num_ops":1,"prepare.queue_time_us":188771,"prepare.run_cpu_time_us":2102,"prepare.run_wall_time_us":82911,"raft.queue_time_us":3476,"raft.run_cpu_time_us":783,"raft.run_wall_time_us":56016,"replication_time_us":166095}],["op",{"apply.queue_time_us":212,"apply.run_cpu_time_us":1501,"apply.run_wall_time_us":1501,"num_ops":1,"prepare.queue_time_us":73469,"prepare.run_cpu_time_us":2212,"prepare.run_wall_time_us":28521,"raft.queue_time_us":4609,"raft.run_cpu_time_us":721,"raft.run_wall_time_us":723,"replication_time_us":17980}]]}
I20260923 17:31:16.346480 17143 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:16.346901 17143 ts_tablet_manager.cc:1397] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
W20260923 17:31:16.339814 16379 rpcz_store.cc:269] 0923 17:31:13.539851 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:13.540009 (+ 158us) service_pool.cc:227] Handling call
0923 17:31:13.588920 (+ 48911us) spinlock_profiling.cc:243] Waited 46.1 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:16.324523 (+2735603us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:15.407561 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:15.407923 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:15.407946 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:15.407957 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:15.412980 (+ 5023us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:15.413023 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:15.413043 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:15.413328 (+ 285us) tablet.cc:699] Row locks acquired
0923 17:31:15.413343 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:15.413457 (+ 114us) write_op.cc:270] Start()
0923 17:31:15.413537 (+ 80us) write_op.cc:276] Timestamp: P: 1790184675413352 usec, L: 0
0923 17:31:15.413552 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:15.444539 (+ 30987us) log.cc:842] Serialized 919 byte log entry
0923 17:31:15.452818 (+ 8279us) op_driver.cc:444] REPLICATION: finished
0923 17:31:15.453210 (+ 392us) write_op.cc:301] APPLY: starting
0923 17:31:15.453280 (+ 70us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:15.453392 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:15.453484 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:15.453759 (+ 275us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:15.453901 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:15.453941 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:15.454130 (+ 189us) log.cc:842] Serialized 29 byte log entry
0923 17:31:15.455155 (+ 1025us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:15.455242 (+ 87us) write_op.cc:454] Released schema lock
0923 17:31:15.455291 (+ 49us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:15.455471 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:15.455492 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"spinlock_wait_cycles":105964416,"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":2465,"apply.run_wall_time_us":2464,"num_ops":2,"prepare.queue_time_us":127089,"prepare.run_cpu_time_us":2993,"prepare.run_wall_time_us":56977,"raft.queue_time_us":78,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":624,"replication_time_us":39186}]]}
I20260923 17:31:16.351547 16379 master_service.cc:946] ReplaceTablet: received request to replace tablet a60bf118082b4e9991bfeb3ac59eb118 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:16.356751 17143 raft_consensus.cc:348] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.357388 17143 raft_consensus.cc:374] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.357621 17143 raft_consensus.cc:729] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.358182 17143 consensus_queue.cc:260] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.361600 17146 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:16.365461 17146 ts_tablet_manager.cc:1397] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.073s user 0.018s sys 0.000s
I20260923 17:31:16.368119 17146 raft_consensus.cc:348] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.375233 17146 raft_consensus.cc:374] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.375557 17146 raft_consensus.cc:729] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.377110 17146 consensus_queue.cc:260] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.380206 17146 ts_tablet_manager.cc:1428] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260923 17:31:16.381881 17146 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:16.381938 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:16.385437 16972 leader_election.cc:341] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.385908 16972 leader_election.cc:302] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:16.372942 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.387142 17466 raft_consensus.cc:2726] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:16.387836 17146 tablet_bootstrap.cc:654] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.379602 17285 leader_election.cc:288] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.397547 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:16.398586 17040 raft_consensus.cc:2454] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:16.402419 17454 tablet.cc:1717] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Skipping requested alter to schema version 2, tablet already version 2
I20260923 17:31:16.406785 17471 ts_tablet_manager.cc:1918] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:16.407133 17471 log.cc:1196] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/21a629ce6ab14d12b6a83ce6bc56f70f
I20260923 17:31:16.407786 17471 ts_tablet_manager.cc:1939] T 21a629ce6ab14d12b6a83ce6bc56f70f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:16.413517 17143 ts_tablet_manager.cc:1428] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.066s user 0.007s sys 0.000s
I20260923 17:31:16.414709 17143 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:16.431682 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 21a629ce6ab14d12b6a83ce6bc56f70f (table default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]) successfully deleted
I20260923 17:31:16.433692 17471 tablet_replica.cc:330] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:16.436239 17144 tablet_bootstrap.cc:492] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:16.436719 17144 ts_tablet_manager.cc:1397] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.497s user 0.020s sys 0.007s
I20260923 17:31:16.439138 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.440884 17146 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:16.441250 17146 ts_tablet_manager.cc:1397] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.060s user 0.017s sys 0.032s
I20260923 17:31:16.443333 17146 raft_consensus.cc:348] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.443974 17146 raft_consensus.cc:374] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.444203 17146 raft_consensus.cc:729] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.445036 17146 consensus_queue.cc:260] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.446599 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:16.448879 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.450275 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.450932 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.452085 17146 ts_tablet_manager.cc:1428] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.011s user 0.001s sys 0.004s
I20260923 17:31:16.452623 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.439857 17143 tablet_bootstrap.cc:654] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.449646 17405 raft_consensus.cc:482] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.453003 17146 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:16.453270 17405 raft_consensus.cc:504] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.454540 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fc7428f55041b6b939a8ca08574b90 with delete_type TABLET_DATA_DELETED (replaced by tablet e818f6192de6472cb27747c0e3a1f128) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.455160 17405 leader_election.cc:288] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.455502 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.455924 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:16.456620 16743 raft_consensus.cc:2454] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:16.456804 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.457788 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet ceb787c0863847289400e62718d218a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:16.457957 16826 leader_election.cc:302] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.459004 17146 tablet_bootstrap.cc:654] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.459262 17405 raft_consensus.cc:2781] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:16.459582 17405 raft_consensus.cc:482] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:16.459846 17405 raft_consensus.cc:3037] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.461400 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:16.462124 17040 raft_consensus.cc:2454] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:16.464633 17471 raft_consensus.cc:2229] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:16.465624 17471 raft_consensus.cc:2258] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.468922 17144 raft_consensus.cc:348] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.469854 17144 raft_consensus.cc:374] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.470152 17144 raft_consensus.cc:729] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:16.476050 16718 consensus_meta.cc:287] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.505s user 0.008s sys 0.000s
I20260923 17:31:16.386868 17427 raft_consensus.cc:504] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.492376 17471 ts_tablet_manager.cc:1905] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:16.493831 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:16.494493 16890 raft_consensus.cc:3037] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.502615 16890 raft_consensus.cc:2454] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:16.504153 16677 leader_election.cc:302] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.505095 17473 raft_consensus.cc:2781] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:16.505520 17473 raft_consensus.cc:686] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:16.507354 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_4c7c906814484e6e95b4066c4ce97fb7" } modify_external_catalogs: true
I20260923 17:31:16.507879 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_4c7c906814484e6e95b4066c4ce97fb7" } modify_external_catalogs: true
I20260923 17:31:16.510170 17143 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:16.510691 17143 ts_tablet_manager.cc:1397] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.000s
I20260923 17:31:16.511155 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_f8418b500b374960b37e876bfc1b7a65" } new_table_name: "default.Table_4bd2aa1e022c48b7841a7aae1ba594a4" modify_external_catalogs: true
I20260923 17:31:16.513114 17143 raft_consensus.cc:348] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.513727 17143 raft_consensus.cc:374] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.513963 17143 raft_consensus.cc:729] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.515606 17143 consensus_queue.cc:260] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.515846 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_363f829391944ba18bdfc4f2e1f34dce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:16.517606 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_363f829391944ba18bdfc4f2e1f34dce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.518491 17143 ts_tablet_manager.cc:1428] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260923 17:31:16.520066 17143 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:16.521016 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_dc1abdd9ffc44b1d83f6e604bec652df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:16.522465 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc1abdd9ffc44b1d83f6e604bec652df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.532930 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_5b64b645bddd49ec8eb278fa41130724"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:16.534853 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b64b645bddd49ec8eb278fa41130724 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.545796 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:16.514647 17144 consensus_queue.cc:260] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.506290 17473 consensus_queue.cc:237] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.552690 17343 ts_tablet_manager.cc:1918] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:16.554049 17143 tablet_bootstrap.cc:654] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:16.557996 16722 consensus_meta.cc:287] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.506s user 0.004s sys 0.003s
I20260923 17:31:16.560123 17427 leader_election.cc:288] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.566238 17144 ts_tablet_manager.cc:1428] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.129s user 0.006s sys 0.000s
I20260923 17:31:16.567142 17144 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:16.479866 17405 raft_consensus.cc:504] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.569867 17405 leader_election.cc:288] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:16.566977 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 166760us
I20260923 17:31:16.573486 17403 ts_tablet_manager.cc:1918] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:16.574033 17403 log.cc:1196] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/0721fa178c0347598351b9d3512089b8
I20260923 17:31:16.573958 17427 consensus_queue.cc:1035] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:16.574928 17403 ts_tablet_manager.cc:1939] T 0721fa178c0347598351b9d3512089b8 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:16.575047 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:16.575716 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 82) took 1962 ms. Trace:
I20260923 17:31:16.576010 17403 rpcz_store.cc:276] 0923 17:31:14.613492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:14.616421 (+ 2929us) service_pool.cc:227] Handling call
0923 17:31:15.874821 (+1258400us) ts_tablet_manager.cc:1145] Deleting tablet 0721fa178c0347598351b9d3512089b8
0923 17:31:15.874872 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:15.989808 (+114936us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:16.573146 (+583338us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:16.575659 (+ 2513us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40065,"tablet-delete.queue_time_us":1257307}
W20260923 17:31:16.592348 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260923 17:31:16.593461 16885 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s user 0.015s sys 0.004s
W20260923 17:31:16.593901 16885 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s user 0.015s sys 0.004s
I20260923 17:31:16.599109 17471 ts_tablet_manager.cc:1918] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:16.613884 16383 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 144) took 1088 ms. Trace:
I20260923 17:31:16.579118 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:16.615005 16743 raft_consensus.cc:3037] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.617250 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_bcf64eeacb1b4b85afb1ef76d08123f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:16.618781 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcf64eeacb1b4b85afb1ef76d08123f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.622396 16743 raft_consensus.cc:2454] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:16.615042 16383 rpcz_store.cc:276] 0923 17:31:15.525537 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.307864 (+782327us) service_pool.cc:227] Handling call
0923 17:31:16.515648 (+207784us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df156 00007f912711c034 00007f912711bfc2 00007f912711bef1 00007f912711be6c 00007f912711b122 00007f91253be94c 00007f91253bdd86
0923 17:31:16.518000 (+ 2352us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:16.538918 (+ 20918us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:16.613101 (+ 74183us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:16.613493 (+ 392us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:16.613803 (+ 310us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.541616 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.541928 (+ 312us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.541949 (+ 21us) write_op.cc:435] Acquired schema lock
0923 17:31:16.541960 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:16.542192 (+ 232us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.542223 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.542243 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:16.542413 (+ 170us) tablet.cc:699] Row locks acquired
0923 17:31:16.542426 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.542522 (+ 96us) write_op.cc:270] Start()
0923 17:31:16.542606 (+ 84us) write_op.cc:276] Timestamp: P: 1790184676537078 usec, L: 0
0923 17:31:16.542622 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.543019 (+ 397us) log.cc:842] Serialized 433 byte log entry
0923 17:31:16.610886 (+ 67867us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.611302 (+ 416us) write_op.cc:301] APPLY: starting
0923 17:31:16.611362 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.611467 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.611566 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.611988 (+ 422us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.612126 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:16.612178 (+ 52us) write_op.cc:312] APPLY: finished
0923 17:31:16.612355 (+ 177us) log.cc:842] Serialized 29 byte log entry
0923 17:31:16.612496 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.612561 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:16.612593 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.612764 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.612789 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":474481664,"child_traces":[["op",{"apply.queue_time_us":287,"apply.run_cpu_time_us":1647,"apply.run_wall_time_us":2210,"num_ops":2,"prepare.queue_time_us":1545,"prepare.run_cpu_time_us":1589,"prepare.run_wall_time_us":1588,"replication_time_us":68181}]]}
I20260923 17:31:16.587742 17285 raft_consensus.cc:482] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.625152 16383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_d0e18da1609f485e9684fb7a0228fc35" } modify_external_catalogs: true
I20260923 17:31:16.625696 16383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_d0e18da1609f485e9684fb7a0228fc35" } modify_external_catalogs: true
I20260923 17:31:16.625931 17471 log.cc:1196] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/8879dee684214c6dac3f9d6642a6d97d
I20260923 17:31:16.625555 17285 raft_consensus.cc:504] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.626816 17471 ts_tablet_manager.cc:1939] T 8879dee684214c6dac3f9d6642a6d97d P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:16.627972 17285 leader_election.cc:288] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.629652 16385 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 143) took 1143 ms. Trace:
I20260923 17:31:16.629561 16826 leader_election.cc:302] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.630757 17413 raft_consensus.cc:2781] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:16.631233 17144 tablet_bootstrap.cc:654] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:16.631161 17413 raft_consensus.cc:686] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:16.632277 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 85) took 1166 ms. Trace:
I20260923 17:31:16.631897 17413 consensus_queue.cc:237] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.630048 16385 rpcz_store.cc:276] 0923 17:31:15.486463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.274731 (+788268us) service_pool.cc:227] Handling call
0923 17:31:16.532701 (+257970us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df156 00007f912711c034 00007f912711bfc2 00007f912711bef1 00007f912711be6c 00007f912711b122 00007f91253be94c 00007f91253bdd86
0923 17:31:16.535346 (+ 2645us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:16.536800 (+ 1454us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:16.628344 (+ 91544us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:16.628646 (+ 302us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:16.629591 (+ 945us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.543287 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.543610 (+ 323us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.543629 (+ 19us) write_op.cc:435] Acquired schema lock
0923 17:31:16.543642 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:16.543883 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.543914 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.543935 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:16.544106 (+ 171us) tablet.cc:699] Row locks acquired
0923 17:31:16.544119 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.544207 (+ 88us) write_op.cc:270] Start()
0923 17:31:16.548510 (+ 4303us) write_op.cc:276] Timestamp: P: 1790184676538766 usec, L: 0
0923 17:31:16.548528 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.548957 (+ 429us) log.cc:842] Serialized 433 byte log entry
0923 17:31:16.611117 (+ 62160us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.613679 (+ 2562us) write_op.cc:301] APPLY: starting
0923 17:31:16.613735 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.613844 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.614139 (+ 295us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.614519 (+ 380us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.614658 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:16.614703 (+ 45us) write_op.cc:312] APPLY: finished
0923 17:31:16.614940 (+ 237us) log.cc:842] Serialized 29 byte log entry
0923 17:31:16.615800 (+ 860us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.615875 (+ 75us) write_op.cc:454] Released schema lock
0923 17:31:16.615907 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.616185 (+ 278us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.616211 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":535690496,"child_traces":[["op",{"apply.queue_time_us":2392,"apply.run_cpu_time_us":1723,"apply.run_wall_time_us":2689,"num_ops":2,"prepare.queue_time_us":2680,"prepare.run_cpu_time_us":1666,"prepare.run_wall_time_us":5842,"replication_time_us":62518}]]}
I20260923 17:31:16.633406 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:16.634016 16890 raft_consensus.cc:2454] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:16.635470 16677 leader_election.cc:302] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.636255 17285 raft_consensus.cc:2781] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:16.636624 17285 raft_consensus.cc:482] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:16.636804 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 0721fa178c0347598351b9d3512089b8 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]) successfully deleted
I20260923 17:31:16.636972 17285 raft_consensus.cc:3037] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.637389 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 8879dee684214c6dac3f9d6642a6d97d (table default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]) successfully deleted
I20260923 17:31:16.570966 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723000dc175844e990740dc503b53ced" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:16.638053 17040 raft_consensus.cc:3037] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.632602 17403 rpcz_store.cc:276] 0923 17:31:15.465665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:15.466924 (+ 1259us) service_pool.cc:227] Handling call
0923 17:31:16.631874 (+1164950us) ts_tablet_manager.cc:1145] Deleting tablet f1ccdf1bd7a64f51965e0a6dc7e033c9
0923 17:31:16.631923 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:16.632196 (+ 273us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1149930}
W20260923 17:31:16.644650 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 already in progress: creating tablet
I20260923 17:31:16.645481 17040 raft_consensus.cc:2454] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:16.645844 17466 raft_consensus.cc:482] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.646418 17343 log.cc:1196] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/2c842eff86634bf4bc70c426b5226d66
W20260923 17:31:16.646667 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 2c842eff86634bf4bc70c426b5226d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c842eff86634bf4bc70c426b5226d66 already in progress: creating tablet
I20260923 17:31:16.646344 17466 raft_consensus.cc:504] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.647346 17343 ts_tablet_manager.cc:1939] T 2c842eff86634bf4bc70c426b5226d66 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:16.649190 17466 leader_election.cc:288] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:16.650661 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:16.652017 16972 leader_election.cc:341] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.652341 16385 master_service.cc:946] ReplaceTablet: received request to replace tablet ff22ffe24d424b1e8659579febfe1270 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:16.659107 17285 raft_consensus.cc:504] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.665025 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bcaddfad4f45739a705c17bdfe332f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:16.665984 17040 raft_consensus.cc:3037] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.673343 17454 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:16.663409 17403 tablet_replica.cc:330] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:16.664191 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:16.690302 16972 leader_election.cc:341] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.690754 16972 leader_election.cc:302] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:16.691959 17466 raft_consensus.cc:2726] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:16.694139 17427 consensus_queue.cc:1035] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:16.694828 17285 leader_election.cc:288] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.700904 17471 tablet_replica.cc:330] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:16.701910 17471 raft_consensus.cc:2229] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:16.702966 17471 pending_rounds.cc:62] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Trying to abort 1 pending ops.
I20260923 17:31:16.703295 17471 pending_rounds.cc:68] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596422233341952 op_type: NO_OP noop_request { }
I20260923 17:31:16.703819 17471 raft_consensus.cc:2866] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:16.705742 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:16.706642 17036 raft_consensus.cc:2454] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:16.708475 17471 raft_consensus.cc:2258] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.708874 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:16.709949 16890 raft_consensus.cc:3037] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.718318 16890 raft_consensus.cc:2454] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:16.720559 17403 raft_consensus.cc:2229] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:16.722807 17403 pending_rounds.cc:62] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:16.723060 17403 pending_rounds.cc:68] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596435198533632 op_type: NO_OP noop_request { }
I20260923 17:31:16.723443 17403 raft_consensus.cc:2866] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:16.723759 17403 raft_consensus.cc:2258] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.736982 16677 leader_election.cc:302] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:16.738166 17285 raft_consensus.cc:2781] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:16.738660 17285 raft_consensus.cc:686] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:16.739398 17285 consensus_queue.cc:237] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.747032 17471 ts_tablet_manager.cc:1905] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:16.754309 17486 consensus_queue.cc:1159] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be93fda0843b44f5abc49e6f35c69217" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260923 17:31:16.760833 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "456273cf57314cb8a05c8e765543913f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:16.761380 17041 raft_consensus.cc:3037] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.759991 17430 raft_consensus.cc:482] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.761787 17430 raft_consensus.cc:504] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.763650 17430 leader_election.cc:288] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.770604 17041 raft_consensus.cc:2454] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:16.783067 17405 raft_consensus.cc:482] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.783511 17405 raft_consensus.cc:504] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.784512 17430 consensus_queue.cc:1035] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:16.785570 17405 leader_election.cc:288] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.787034 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_b6843cebc625435d8e90cf414884277d" } new_table_name: "default.Table_3880da019d614925ba87aa4c506e1107" modify_external_catalogs: true
W20260923 17:31:16.796157 16827 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
W20260923 17:31:16.804270 16678 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540
W20260923 17:31:16.809546 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a6894a4ee5fe4884b5162d8055e766fe has no consensus state
I20260923 17:31:16.810822 17405 raft_consensus.cc:482] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.811329 17405 raft_consensus.cc:504] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.814087 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff22ffe24d424b1e8659579febfe1270" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:16.815055 17405 leader_election.cc:288] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.816887 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:16.817741 17041 raft_consensus.cc:2454] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:16.818797 16827 leader_election.cc:302] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:16.819628 17405 raft_consensus.cc:2781] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:16.820149 17405 raft_consensus.cc:482] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:16.820591 17405 raft_consensus.cc:3037] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.828431 17343 tablet_replica.cc:330] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:16.829105 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9b734b250d4920abbafe76f52ec540" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:16.829834 16885 raft_consensus.cc:2454] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:16.837529 16826 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.837942 16826 leader_election.cc:302] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:16.838989 17332 raft_consensus.cc:2726] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:16.838941 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 2c842eff86634bf4bc70c426b5226d66 (table default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]) successfully deleted
I20260923 17:31:16.841858 17144 tablet_bootstrap.cc:492] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:16.842293 17144 ts_tablet_manager.cc:1397] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.275s user 0.019s sys 0.016s
I20260923 17:31:16.845430 17144 raft_consensus.cc:348] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.846063 17144 raft_consensus.cc:374] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.846278 17144 raft_consensus.cc:729] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.849982 17403 ts_tablet_manager.cc:1905] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:16.827630 17405 raft_consensus.cc:504] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.861370 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:16.862040 17041 raft_consensus.cc:3037] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.866473 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ce218dced6994036a697bb38f509808b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:16.873148 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_bba5b469220340c781960e02845d372e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:16.878123 17146 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:16.878584 17146 ts_tablet_manager.cc:1397] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.426s user 0.106s sys 0.117s
I20260923 17:31:16.879833 17430 consensus_queue.cc:1035] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:16.880890 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce218dced6994036a697bb38f509808b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.880806 17146 raft_consensus.cc:348] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.852926 17143 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:16.881441 17146 raft_consensus.cc:374] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.881670 17143 ts_tablet_manager.cc:1397] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.362s user 0.024s sys 0.011s
I20260923 17:31:16.881681 17146 raft_consensus.cc:729] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.882272 17146 consensus_queue.cc:260] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.882790 17471 ts_tablet_manager.cc:1918] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:16.883625 17040 raft_consensus.cc:2454] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:16.885314 17146 ts_tablet_manager.cc:1428] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260923 17:31:16.889210 17343 raft_consensus.cc:2229] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:16.889900 17146 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:16.894129 17343 raft_consensus.cc:2258] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.899220 17343 ts_tablet_manager.cc:1905] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:16.900004 17144 consensus_queue.cc:260] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.907827 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_cb68aa3dd0ac47d485cd2eb2f7a38165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:16.909380 17144 ts_tablet_manager.cc:1428] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.067s user 0.005s sys 0.001s
W20260923 17:31:16.909566 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb68aa3dd0ac47d485cd2eb2f7a38165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.913691 17144 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:16.913769 17041 raft_consensus.cc:2454] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:16.901499 17146 tablet_bootstrap.cc:654] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:16.916482 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bba5b469220340c781960e02845d372e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:16.892460 17405 leader_election.cc:288] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.897154 17143 raft_consensus.cc:348] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.915274 16827 leader_election.cc:302] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:16.923950 17332 raft_consensus.cc:482] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.925822 17471 log.cc:1196] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:16.926007 17332 raft_consensus.cc:504] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.926666 17471 ts_tablet_manager.cc:1939] T 29d44ee57faf4a1d99e077fd781a519c P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:16.929445 17332 leader_election.cc:288] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:16.929988 17332 raft_consensus.cc:2781] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:16.931564 17332 raft_consensus.cc:686] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:16.933400 17332 consensus_queue.cc:237] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.942205 17143 raft_consensus.cc:374] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:16.942603 17143 raft_consensus.cc:729] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:16.944026 17143 consensus_queue.cc:260] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.949337 17143 ts_tablet_manager.cc:1428] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.067s user 0.006s sys 0.000s
W20260923 17:31:16.950058 17498 log.cc:924] Time spent T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260923 17:31:16.952114 17143 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:16.955845 17466 raft_consensus.cc:482] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:16.956489 17466 raft_consensus.cc:504] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:16.958436 17466 leader_election.cc:288] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:16.960255 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:16.960305 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:16.955380 16827 leader_election.cc:341] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 1483835b2bc64873a3570ad17a079713
W20260923 17:31:16.961628 16972 leader_election.cc:341] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:16.962255 16972 leader_election.cc:341] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.962582 16972 leader_election.cc:302] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:16.963708 17466 raft_consensus.cc:2726] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:16.966660 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 29d44ee57faf4a1d99e077fd781a519c (table default.Table_793e622592644e9daae1b9dba1524dba [id=352d5ecd536b4754a5b0bebe1afe0099]) successfully deleted
I20260923 17:31:16.969877 17471 tablet_replica.cc:330] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:16.971469 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:16.974852 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e818f6192de6472cb27747c0e3a1f128" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:16.976207 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1483835b2bc64873a3570ad17a079713" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:16.976802 16743 raft_consensus.cc:2454] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
W20260923 17:31:16.981644 16826 leader_election.cc:341] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.976529 17471 raft_consensus.cc:2229] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 17:31:16.983999 16826 leader_election.cc:341] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:16.985208 17471 raft_consensus.cc:2258] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:16.986382 16826 leader_election.cc:302] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:16.987092 16826 raft_consensus.cc:2670] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:16.991003 17146 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:16.991411 17146 ts_tablet_manager.cc:1397] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.102s user 0.015s sys 0.011s
I20260923 17:31:16.992501 16677 leader_election.cc:302] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:16.996124 17429 raft_consensus.cc:2781] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:16.996505 17429 raft_consensus.cc:482] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:16.996902 17429 raft_consensus.cc:3037] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:16.996732 17471 ts_tablet_manager.cc:1905] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:17.012063 17146 raft_consensus.cc:348] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.017441 17146 raft_consensus.cc:374] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.017688 17146 raft_consensus.cc:729] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.027103 17430 consensus_queue.cc:1035] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:16.994549 17144 tablet_bootstrap.cc:654] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.018265 17146 consensus_queue.cc:260] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.033640 17146 ts_tablet_manager.cc:1428] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.042s user 0.005s sys 0.001s
I20260923 17:31:17.034250 17146 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.055161 17146 tablet_bootstrap.cc:654] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.007225 17429 raft_consensus.cc:504] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.072995 17143 tablet_bootstrap.cc:654] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.079149 17471 ts_tablet_manager.cc:1918] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:17.088651 17471 log.cc:1196] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/1483835b2bc64873a3570ad17a079713
I20260923 17:31:17.090588 17471 ts_tablet_manager.cc:1939] T 1483835b2bc64873a3570ad17a079713 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:17.094524 17430 raft_consensus.cc:482] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.095006 17430 raft_consensus.cc:504] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.095810 17429 leader_election.cc:288] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.096869 17430 leader_election.cc:288] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.098948 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 1483835b2bc64873a3570ad17a079713 (table default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]) successfully deleted
I20260923 17:31:17.099449 17471 tablet_replica.cc:330] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:17.100265 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f124169d87c240a5bd544d3b6693323f because the tablet was not found. No further retry: Not found: Tablet not found: f124169d87c240a5bd544d3b6693323f
W20260923 17:31:17.100257 16678 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
W20260923 17:31:17.100831 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 0721fa178c0347598351b9d3512089b8 because the tablet was not found. No further retry: Not found: Tablet not found: 0721fa178c0347598351b9d3512089b8
W20260923 17:31:17.100862 16678 leader_election.cc:341] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540
W20260923 17:31:17.104532 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_3880da019d614925ba87aa4c506e1107 [id=02fecf5168494501aca594904d9e933e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a6894a4ee5fe4884b5162d8055e766fe has no consensus state
I20260923 17:31:17.105644 17144 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:17.106040 17144 ts_tablet_manager.cc:1397] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.193s user 0.004s sys 0.004s
I20260923 17:31:17.108312 17144 raft_consensus.cc:348] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.109794 17144 raft_consensus.cc:374] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.110435 17144 raft_consensus.cc:729] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.110991 17144 consensus_queue.cc:260] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.117404 17343 ts_tablet_manager.cc:1918] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:17.119575 17343 log.cc:1196] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:17.120270 17343 ts_tablet_manager.cc:1939] T a3bcaddfad4f45739a705c17bdfe332f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:17.121879 17471 raft_consensus.cc:2229] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:17.122617 17144 ts_tablet_manager.cc:1428] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260923 17:31:17.123124 17471 raft_consensus.cc:2258] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:17.123492 17144 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:17.125828 17471 ts_tablet_manager.cc:1905] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:17.128005 17343 tablet_replica.cc:330] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:17.128800 17343 raft_consensus.cc:2229] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:17.129328 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a3bcaddfad4f45739a705c17bdfe332f (table default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]) successfully deleted
W20260923 17:31:17.130339 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 0721fa178c0347598351b9d3512089b8 because the tablet was not found. No further retry: Not found: Tablet not found: 0721fa178c0347598351b9d3512089b8
I20260923 17:31:17.132929 17343 raft_consensus.cc:2258] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:17.141801 17343 ts_tablet_manager.cc:1905] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:17.151351 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 56991.3us
I20260923 17:31:17.154227 17144 tablet_bootstrap.cc:654] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.154080 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9b734b250d4920abbafe76f52ec540" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:17.154688 16885 raft_consensus.cc:3037] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.163177 16885 raft_consensus.cc:2454] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:17.171280 17144 tablet_bootstrap.cc:492] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:17.171701 17144 ts_tablet_manager.cc:1397] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.008s
I20260923 17:31:17.186522 17144 raft_consensus.cc:348] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.187274 17144 raft_consensus.cc:374] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.187518 17144 raft_consensus.cc:729] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.192171 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:17.193516 17466 raft_consensus.cc:482] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.194003 17466 raft_consensus.cc:504] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.194747 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:17.202147 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:17.204638 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:17.207494 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff22ffe24d424b1e8659579febfe1270" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:17.209069 16972 leader_election.cc:341] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.208847 16885 raft_consensus.cc:2454] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:17.210299 16972 leader_election.cc:341] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.210724 16972 leader_election.cc:302] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:17.195981 17466 leader_election.cc:288] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:17.211884 17466 raft_consensus.cc:2726] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:17.215250 17144 consensus_queue.cc:260] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.218920 17143 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:17.219372 17143 ts_tablet_manager.cc:1397] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.267s user 0.027s sys 0.003s
I20260923 17:31:17.221338 17144 ts_tablet_manager.cc:1428] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.045s user 0.006s sys 0.001s
I20260923 17:31:17.221871 17143 raft_consensus.cc:348] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.222889 17143 raft_consensus.cc:374] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.223297 17143 raft_consensus.cc:729] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.226840 17144 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:17.229343 16677 leader_election.cc:302] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:17.233170 17343 ts_tablet_manager.cc:1918] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:17.233662 17343 log.cc:1196] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/723000dc175844e990740dc503b53ced
I20260923 17:31:17.234500 17343 ts_tablet_manager.cc:1939] T 723000dc175844e990740dc503b53ced P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:17.235167 17332 raft_consensus.cc:482] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.235636 17332 raft_consensus.cc:504] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.236502 17471 ts_tablet_manager.cc:1918] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:17.236881 17471 log.cc:1196] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:17.237484 17471 ts_tablet_manager.cc:1939] T a3bcaddfad4f45739a705c17bdfe332f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:17.237614 17332 leader_election.cc:288] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.238255 17146 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:17.238665 17146 ts_tablet_manager.cc:1397] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.205s user 0.027s sys 0.037s
W20260923 17:31:17.240549 16827 leader_election.cc:341] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:17.241175 17285 raft_consensus.cc:2781] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:17.241039 17146 raft_consensus.cc:348] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.241663 17146 raft_consensus.cc:374] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.241883 17146 raft_consensus.cc:729] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.242525 17146 consensus_queue.cc:260] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.255157 17146 ts_tablet_manager.cc:1428] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260923 17:31:17.262683 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 723000dc175844e990740dc503b53ced (table default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]) successfully deleted
W20260923 17:31:17.251925 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 47562us
I20260923 17:31:17.224324 17143 consensus_queue.cc:260] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.265694 17146 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.264357 17471 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 93) took 1031 ms. Trace:
I20260923 17:31:17.267155 17471 rpcz_store.cc:276] 0923 17:31:16.232572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.574869 (+342297us) service_pool.cc:227] Handling call
0923 17:31:17.099303 (+524434us) ts_tablet_manager.cc:1145] Deleting tablet a3bcaddfad4f45739a705c17bdfe332f
0923 17:31:17.099340 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:17.151202 (+ 51862us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:17.236261 (+ 85059us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:17.264292 (+ 28031us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":523684}
I20260923 17:31:17.270752 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a3bcaddfad4f45739a705c17bdfe332f (table default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]) successfully deleted
I20260923 17:31:17.278985 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 2 got 0
I20260923 17:31:17.280423 17146 tablet_bootstrap.cc:654] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.280902 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 151184083ebb4ae4bf24fe171d33cfab (DEFAULT_TABLE table=default.Table_4b6521ef83e24b43a74029fd0c6e7326 [id=756b68953aae4249ad959a37ef082fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.282048 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151184083ebb4ae4bf24fe171d33cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.283187 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.284103 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0c75099d4d43339e88bf7527b10af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.285753 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 151184083ebb4ae4bf24fe171d33cfab (DEFAULT_TABLE table=default.Table_4b6521ef83e24b43a74029fd0c6e7326 [id=756b68953aae4249ad959a37ef082fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.286827 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151184083ebb4ae4bf24fe171d33cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.292023 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:17.296056 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f]). Expected version 1 got 0
I20260923 17:31:17.299081 17143 ts_tablet_manager.cc:1428] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.079s user 0.008s sys 0.000s
I20260923 17:31:17.299787 16379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:17.302088 17143 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:17.302932 17146 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:17.303526 17146 ts_tablet_manager.cc:1397] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.038s user 0.007s sys 0.004s
I20260923 17:31:17.307663 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:17.309720 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a60bf118082b4e9991bfeb3ac59eb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a60bf118082b4e9991bfeb3ac59eb118 already in progress: creating tablet
I20260923 17:31:17.310489 17146 raft_consensus.cc:348] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.311106 17146 raft_consensus.cc:374] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.311316 17146 raft_consensus.cc:729] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.311327 17013 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.311975 17146 consensus_queue.cc:260] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.313035 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0c75099d4d43339e88bf7527b10af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.312816 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 14bec348cce841a689f69bacdf21004d (DEFAULT_TABLE table=default.Table_6691348cb752455e82da3fd750e9a52b [id=fce4d77a4f83493e91775d2292893116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.313813 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bec348cce841a689f69bacdf21004d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.317845 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 3208c90d96f349988acb6705f2f438d5 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.318602 17146 ts_tablet_manager.cc:1428] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20260923 17:31:17.318974 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3208c90d96f349988acb6705f2f438d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.319501 17146 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.327736 17144 tablet_bootstrap.cc:654] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.332784 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 14bec348cce841a689f69bacdf21004d (DEFAULT_TABLE table=default.Table_6691348cb752455e82da3fd750e9a52b [id=fce4d77a4f83493e91775d2292893116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.333956 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bec348cce841a689f69bacdf21004d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.334718 17146 tablet_bootstrap.cc:654] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.340612 16379 catalog_manager.cc:6565] ReplaceTablet: tablet a60bf118082b4e9991bfeb3ac59eb118 deleted and replaced by tablet 261d1d03290546969a74ca60beb890eb
I20260923 17:31:17.341104 16379 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 148) took 1307 ms. Trace:
I20260923 17:31:17.325126 17285 raft_consensus.cc:686] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:17.355687 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 151184083ebb4ae4bf24fe171d33cfab (DEFAULT_TABLE table=default.Table_4b6521ef83e24b43a74029fd0c6e7326 [id=756b68953aae4249ad959a37ef082fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.356947 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.358109 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0c75099d4d43339e88bf7527b10af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.360481 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:17.361675 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 14bec348cce841a689f69bacdf21004d (DEFAULT_TABLE table=default.Table_6691348cb752455e82da3fd750e9a52b [id=fce4d77a4f83493e91775d2292893116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.362742 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bec348cce841a689f69bacdf21004d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.365427 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:17.364274 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 3208c90d96f349988acb6705f2f438d5 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.366757 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3208c90d96f349988acb6705f2f438d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.341557 16379 rpcz_store.cc:276] 0923 17:31:16.033299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.351379 (+318080us) service_pool.cc:227] Handling call
0923 17:31:16.510370 (+158991us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295e1b81 00007f912713cd37 00007f912713ccc2 00007f912713cbf1 00007f912713cb6c 00007f912713be22 00007f91253be94c 00007f91253bdd86
0923 17:31:17.341028 (+830658us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:17.143650 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:17.144015 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:17.144038 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:17.144049 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:17.144406 (+ 357us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:17.144453 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:17.144479 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:17.144716 (+ 237us) tablet.cc:699] Row locks acquired
0923 17:31:17.144730 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:17.144859 (+ 129us) write_op.cc:270] Start()
0923 17:31:17.144950 (+ 91us) write_op.cc:276] Timestamp: P: 1790184677144693 usec, L: 0
0923 17:31:17.144968 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:17.145501 (+ 533us) log.cc:842] Serialized 631 byte log entry
0923 17:31:17.169904 (+ 24403us) op_driver.cc:444] REPLICATION: finished
0923 17:31:17.174407 (+ 4503us) write_op.cc:301] APPLY: starting
0923 17:31:17.174465 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:17.174580 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:17.174688 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:17.175050 (+ 362us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:17.175249 (+ 199us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:17.175289 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:17.175478 (+ 189us) log.cc:842] Serialized 29 byte log entry
0923 17:31:17.176106 (+ 628us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.176182 (+ 76us) write_op.cc:454] Released schema lock
0923 17:31:17.176213 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:17.179311 (+ 3098us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.179365 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":361220480,"child_traces":[["op",{"apply.queue_time_us":4329,"apply.run_cpu_time_us":1873,"apply.run_wall_time_us":5242,"num_ops":2,"prepare.queue_time_us":236,"prepare.run_cpu_time_us":2237,"prepare.run_wall_time_us":5344,"raft.queue_time_us":159,"raft.run_cpu_time_us":725,"raft.run_wall_time_us":726,"replication_time_us":24842}]]}
I20260923 17:31:17.369246 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151184083ebb4ae4bf24fe171d33cfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.371436 16677 leader_election.cc:302] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:17.372521 17430 raft_consensus.cc:2781] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:17.372825 17430 raft_consensus.cc:482] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:17.373111 17430 raft_consensus.cc:3037] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.388487 16723 tablet_service.cc:1467] Processing CreateTablet for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (DEFAULT_TABLE table=default.Table_ed9555c29504496aac9478914859831b [id=8abd6be5200d427c80760d3772ccabce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.389693 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53fd73fce64b658b09a0fbfcdbb0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.396643 16867 tablet_service.cc:1467] Processing CreateTablet for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (DEFAULT_TABLE table=default.Table_ed9555c29504496aac9478914859831b [id=8abd6be5200d427c80760d3772ccabce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.397871 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53fd73fce64b658b09a0fbfcdbb0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.401156 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 373ad39e4d134ffc80caa73a83b80795 (DEFAULT_TABLE table=default.Table_242628c220d447349a2bd325fe77290b [id=406d0c10dfc14cc08a5a81ccdb0ebe4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.402267 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 373ad39e4d134ffc80caa73a83b80795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.404017 16388 catalog_manager.cc:5495] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:17.405297 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_f8418b500b374960b37e876bfc1b7a65 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 2 got 0
I20260923 17:31:17.406659 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 373ad39e4d134ffc80caa73a83b80795 (DEFAULT_TABLE table=default.Table_242628c220d447349a2bd325fe77290b [id=406d0c10dfc14cc08a5a81ccdb0ebe4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.407786 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 373ad39e4d134ffc80caa73a83b80795. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:17.413125 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 62872.1us
I20260923 17:31:17.414744 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389ecb50e0c34b4d92b5296bb17f4ab1" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:17.415460 16743 raft_consensus.cc:2454] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:17.417447 16826 leader_election.cc:302] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:17.418587 17332 raft_consensus.cc:2781] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:17.418902 17332 raft_consensus.cc:482] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:17.419204 17332 raft_consensus.cc:3037] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.431293 16725 tablet_service.cc:1467] Processing CreateTablet for tablet a6894a4ee5fe4884b5162d8055e766fe (DEFAULT_TABLE table=default.Table_3880da019d614925ba87aa4c506e1107 [id=02fecf5168494501aca594904d9e933e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.432497 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6894a4ee5fe4884b5162d8055e766fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.432670 17285 consensus_queue.cc:237] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.441238 16871 tablet_service.cc:1467] Processing CreateTablet for tablet a6894a4ee5fe4884b5162d8055e766fe (DEFAULT_TABLE table=default.Table_3880da019d614925ba87aa4c506e1107 [id=02fecf5168494501aca594904d9e933e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.442600 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6894a4ee5fe4884b5162d8055e766fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.445319 17343 tablet_replica.cc:330] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:17.446262 17343 raft_consensus.cc:2229] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:17.446764 17343 raft_consensus.cc:2258] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:17.449630 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 3208c90d96f349988acb6705f2f438d5 (DEFAULT_TABLE table=default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.450847 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3208c90d96f349988acb6705f2f438d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.457484 17020 tablet_service.cc:1467] Processing CreateTablet for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (DEFAULT_TABLE table=default.Table_ed9555c29504496aac9478914859831b [id=8abd6be5200d427c80760d3772ccabce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.458841 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53fd73fce64b658b09a0fbfcdbb0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.460572 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 373ad39e4d134ffc80caa73a83b80795 (DEFAULT_TABLE table=default.Table_242628c220d447349a2bd325fe77290b [id=406d0c10dfc14cc08a5a81ccdb0ebe4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.461627 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 373ad39e4d134ffc80caa73a83b80795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.465273 17016 tablet_service.cc:1467] Processing CreateTablet for tablet a6894a4ee5fe4884b5162d8055e766fe (DEFAULT_TABLE table=default.Table_3880da019d614925ba87aa4c506e1107 [id=02fecf5168494501aca594904d9e933e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.467270 17016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6894a4ee5fe4884b5162d8055e766fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.471083 17343 ts_tablet_manager.cc:1905] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:17.482110 17473 raft_consensus.cc:482] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.482664 17473 raft_consensus.cc:504] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.489774 17146 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:17.490181 17146 ts_tablet_manager.cc:1397] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.171s user 0.007s sys 0.002s
I20260923 17:31:17.492297 17146 raft_consensus.cc:348] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.492883 17146 raft_consensus.cc:374] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.492990 17473 leader_election.cc:288] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.493113 17146 raft_consensus.cc:729] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.493664 17146 consensus_queue.cc:260] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.495143 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:17.495604 17146 ts_tablet_manager.cc:1428] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:17.495707 17038 raft_consensus.cc:2454] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:17.497193 16678 leader_election.cc:302] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:17.498000 17473 raft_consensus.cc:2781] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:17.498308 17473 raft_consensus.cc:482] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:17.498605 17473 raft_consensus.cc:3037] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.506961 17473 raft_consensus.cc:504] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.508900 17146 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.511495 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:17.462162 17332 raft_consensus.cc:504] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.516253 16716 tablet_service.cc:1467] Processing CreateTablet for tablet e92cfe501505458ea46da095a8c0d30a (DEFAULT_TABLE table=default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.522258 17509 raft_consensus.cc:482] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.522734 17509 raft_consensus.cc:504] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.524580 17509 leader_election.cc:288] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:17.525393 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92cfe501505458ea46da095a8c0d30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.491655 17430 raft_consensus.cc:504] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.533974 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f3ed1bd50d4e2eb30e1a6b6ee8178d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:17.535459 16972 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.539968 17146 tablet_bootstrap.cc:654] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.540750 17430 leader_election.cc:288] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:17.543143 16678 leader_election.cc:341] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:17.539969 17403 ts_tablet_manager.cc:1918] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:17.550415 17403 log.cc:1196] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:17.551508 17403 ts_tablet_manager.cc:1939] T a3bcaddfad4f45739a705c17bdfe332f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:17.555621 16720 tablet_service.cc:1467] Processing CreateTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 (DEFAULT_TABLE table=default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.555665 17013 tablet_service.cc:1467] Processing CreateTablet for tablet e92cfe501505458ea46da095a8c0d30a (DEFAULT_TABLE table=default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.556732 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3504813e9c4698a546b6d6fa0fbcb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.559420 17012 tablet_service.cc:1467] Processing CreateTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 (DEFAULT_TABLE table=default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.560616 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3504813e9c4698a546b6d6fa0fbcb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.573382 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92cfe501505458ea46da095a8c0d30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.558012 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f3ed1bd50d4e2eb30e1a6b6ee8178d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:17.585724 16972 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.586212 16972 leader_election.cc:302] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:17.586912 17509 raft_consensus.cc:2726] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:17.558609 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:17.587404 17146 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:17.560382 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff22ffe24d424b1e8659579febfe1270" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:17.587764 17146 ts_tablet_manager.cc:1397] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.007s
I20260923 17:31:17.588027 16893 raft_consensus.cc:3037] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.562357 17405 raft_consensus.cc:482] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.512580 16388 catalog_manager.cc:5495] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:17.593063 17405 raft_consensus.cc:504] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.593873 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
W20260923 17:31:17.594764 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e818f6192de6472cb27747c0e3a1f128 consensus state has no leader
I20260923 17:31:17.597113 17405 leader_election.cc:288] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.597491 17146 raft_consensus.cc:348] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.598155 17146 raft_consensus.cc:374] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.598402 17146 raft_consensus.cc:729] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.598965 17146 consensus_queue.cc:260] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.604588 16870 tablet_service.cc:1467] Processing CreateTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 (DEFAULT_TABLE table=default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.605676 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db3504813e9c4698a546b6d6fa0fbcb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.605149 16866 tablet_service.cc:1467] Processing CreateTablet for tablet e92cfe501505458ea46da095a8c0d30a (DEFAULT_TABLE table=default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.607380 16382 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 147) took 1691 ms. Trace:
I20260923 17:31:17.607694 16382 rpcz_store.cc:276] 0923 17:31:15.916055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.347942 (+431887us) service_pool.cc:227] Handling call
0923 17:31:16.511044 (+163102us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df7d6 00007f9127125634 00007f91271255c2 00007f91271254f1 00007f912712546c 00007f9127124722 00007f91253be94c 00007f91253bdd86
0923 17:31:16.511781 (+ 737us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:16.512460 (+ 679us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:16.513035 (+ 575us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:16.513228 (+ 193us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:16.605665 (+ 92437us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:17.607300 (+1001635us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.523134 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.523498 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.523520 (+ 22us) write_op.cc:435] Acquired schema lock
0923 17:31:16.523532 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:16.523789 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.523826 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.523849 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:16.524002 (+ 153us) tablet.cc:699] Row locks acquired
0923 17:31:16.524015 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.524130 (+ 115us) write_op.cc:270] Start()
0923 17:31:16.524213 (+ 83us) write_op.cc:276] Timestamp: P: 1790184676518681 usec, L: 0
0923 17:31:16.524283 (+ 70us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.524769 (+ 486us) log.cc:842] Serialized 405 byte log entry
0923 17:31:16.602777 (+ 78008us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.604343 (+ 1566us) write_op.cc:301] APPLY: starting
0923 17:31:16.604384 (+ 41us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.604462 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.604556 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.604679 (+ 123us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.604782 (+ 103us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.604821 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:16.604951 (+ 130us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.605072 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.605117 (+ 45us) write_op.cc:454] Released schema lock
0923 17:31:16.605146 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.605285 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.605310 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":373233920,"child_traces":[["op",{"apply.queue_time_us":1464,"apply.run_cpu_time_us":1105,"apply.run_wall_time_us":1105,"num_ops":1,"prepare.queue_time_us":8750,"prepare.run_cpu_time_us":1788,"prepare.run_wall_time_us":1803,"replication_time_us":78427}]]}
I20260923 17:31:17.612077 17343 ts_tablet_manager.cc:1918] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:17.612429 17343 log.cc:1196] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:17.612521 16893 raft_consensus.cc:2454] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:17.612993 17343 ts_tablet_manager.cc:1939] T a60bf118082b4e9991bfeb3ac59eb118 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:17.613458 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_21e3ee5ab1e640649f6366a299f52319"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:17.615638 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21e3ee5ab1e640649f6366a299f52319 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:17.622926 17332 leader_election.cc:288] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.626868 17473 leader_election.cc:288] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:17.633095 17143 tablet_bootstrap.cc:654] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.630565 16720 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2a0d1f4344ad69eb2c0a4e208839a (DEFAULT_TABLE table=default.Table_41fb14b41ba048a58552a94525c3436f [id=57add86c69d34ebc89de21ad28a61d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.643124 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 96) took 1252 ms. Trace:
I20260923 17:31:17.669361 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2a0d1f4344ad69eb2c0a4e208839a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.669557 17403 rpcz_store.cc:276] 0923 17:31:16.427665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.438936 (+ 11271us) service_pool.cc:227] Handling call
0923 17:31:16.663210 (+224274us) ts_tablet_manager.cc:1145] Deleting tablet a3bcaddfad4f45739a705c17bdfe332f
0923 17:31:16.663267 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:16.722290 (+ 59023us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b5400150754. stack: 00007f24a4f0c033 00007f24a4f0bf30 00007f24a4a8d534 00000000004db2b4 00000000004db249 00007f24abe18471 00007f24abe9b352 00007f24ac2b126c 00007f24acebecfa 00007f24aceeee10 00007f24aced5e0d 00007f24aced5db9 00007f24aced5d49 00007f24aced5d11 00007f24aced4f3d 0000000000580784
0923 17:31:16.863796 (+141506us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:17.539600 (+675804us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:17.643052 (+103452us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":62882048,"tablet-delete.queue_time_us":223129}
I20260923 17:31:17.671170 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 98) took 1222 ms. Trace:
I20260923 17:31:17.671195 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a3bcaddfad4f45739a705c17bdfe332f (table default.Table_ad3626a8c0e24d85a90f0629af2c5b75 [id=53292ad6b8664885bdde6b8bb2789d57]) successfully deleted
I20260923 17:31:17.671471 17403 rpcz_store.cc:276] 0923 17:31:16.448574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.448703 (+ 129us) service_pool.cc:227] Handling call
0923 17:31:17.670911 (+1222208us) ts_tablet_manager.cc:1145] Deleting tablet 29d44ee57faf4a1d99e077fd781a519c
0923 17:31:17.670956 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:17.671118 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1221616}
W20260923 17:31:17.647096 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 81368.8us
W20260923 17:31:17.672866 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 29d44ee57faf4a1d99e077fd781a519c because the tablet was not found. No further retry: Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:17.645927 17144 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:17.675139 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2beb8360d54efeb2fa74c005d011d1" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:17.675664 17144 ts_tablet_manager.cc:1397] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.449s user 0.012s sys 0.007s
I20260923 17:31:17.676419 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389ecb50e0c34b4d92b5296bb17f4ab1" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:17.676908 16743 raft_consensus.cc:3037] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.677860 17144 raft_consensus.cc:348] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.678450 17144 raft_consensus.cc:374] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.678658 17144 raft_consensus.cc:729] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.679199 17144 consensus_queue.cc:260] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.690579 17146 ts_tablet_manager.cc:1428] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.103s user 0.006s sys 0.001s
I20260923 17:31:17.691210 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:17.693837 16743 raft_consensus.cc:2454] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:17.700212 17144 ts_tablet_manager.cc:1428] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
W20260923 17:31:17.700006 16677 leader_election.cc:341] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.701637 16677 leader_election.cc:302] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:17.702651 17509 raft_consensus.cc:482] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.702816 17285 raft_consensus.cc:2781] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:17.702808 17011 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2a0d1f4344ad69eb2c0a4e208839a (DEFAULT_TABLE table=default.Table_41fb14b41ba048a58552a94525c3436f [id=57add86c69d34ebc89de21ad28a61d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.703164 17509 raft_consensus.cc:504] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.703860 17285 raft_consensus.cc:686] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:17.704661 17285 consensus_queue.cc:237] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.705397 17509 leader_election.cc:288] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:17.705884 17015 tablet_service.cc:1467] Processing CreateTablet for tablet e329dd46a3a245b6826b01ac8de56621 (DEFAULT_TABLE table=default.Table_a76ca37973d44e81a5631b2e7dc25739 [id=61954d5e2506486796d76b8d0d1f6d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.711985 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_95f3f4adfca44bc791a565a2d16257f4" } new_table_name: "default.Table_11aad91009934d2fa94936fa3f7b98ba" modify_external_catalogs: true
I20260923 17:31:17.714207 17403 tablet_replica.cc:330] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:17.717470 17403 raft_consensus.cc:2229] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:17.718286 17403 raft_consensus.cc:2258] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:17.717478 17144 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:17.718835 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:17.720269 17403 ts_tablet_manager.cc:1905] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:17.720283 16972 leader_election.cc:341] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.714869 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:17.723469 16972 leader_election.cc:341] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.723910 16972 leader_election.cc:302] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:17.728329 17509 raft_consensus.cc:2726] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:17.730321 16677 leader_election.cc:341] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.732177 16869 tablet_service.cc:1467] Processing CreateTablet for tablet e329dd46a3a245b6826b01ac8de56621 (DEFAULT_TABLE table=default.Table_a76ca37973d44e81a5631b2e7dc25739 [id=61954d5e2506486796d76b8d0d1f6d3f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:17.734685 16826 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.735901 16826 leader_election.cc:302] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:17.737269 17405 consensus_queue.cc:1035] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:17.738601 17405 raft_consensus.cc:2781] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:17.741001 17405 raft_consensus.cc:686] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:17.741750 17405 consensus_queue.cc:237] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.743460 17144 tablet_bootstrap.cc:654] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.749577 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92cfe501505458ea46da095a8c0d30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.750633 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329dd46a3a245b6826b01ac8de56621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.753235 16717 tablet_service.cc:1467] Processing CreateTablet for tablet e329dd46a3a245b6826b01ac8de56621 (DEFAULT_TABLE table=default.Table_a76ca37973d44e81a5631b2e7dc25739 [id=61954d5e2506486796d76b8d0d1f6d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.773089 17144 tablet_bootstrap.cc:492] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:17.776933 17144 ts_tablet_manager.cc:1397] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.002s
I20260923 17:31:17.779183 17144 raft_consensus.cc:348] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.779832 17144 raft_consensus.cc:374] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.779309 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329dd46a3a245b6826b01ac8de56621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.781953 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:17.780089 17144 raft_consensus.cc:729] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.783386 17144 consensus_queue.cc:260] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.784044 16873 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2a0d1f4344ad69eb2c0a4e208839a (DEFAULT_TABLE table=default.Table_41fb14b41ba048a58552a94525c3436f [id=57add86c69d34ebc89de21ad28a61d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:17.785349 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2a0d1f4344ad69eb2c0a4e208839a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.785521 17144 ts_tablet_manager.cc:1428] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260923 17:31:17.797669 17403 ts_tablet_manager.cc:1918] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:17.799867 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:17.800453 17041 raft_consensus.cc:3037] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:17.801005 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2a0d1f4344ad69eb2c0a4e208839a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:17.807238 17146 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.808844 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329dd46a3a245b6826b01ac8de56621. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:17.790467 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 54718.1us
W20260923 17:31:17.802966 16827 leader_election.cc:341] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1
W20260923 17:31:17.818496 16827 consensus_peers.cc:487] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 723000dc175844e990740dc503b53ced. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:17.812786 17041 raft_consensus.cc:2454] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:17.800906 17144 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:17.822052 17403 log.cc:1196] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/8879dee684214c6dac3f9d6642a6d97d
I20260923 17:31:17.823593 17403 ts_tablet_manager.cc:1939] T 8879dee684214c6dac3f9d6642a6d97d P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:17.799853 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a60bf118082b4e9991bfeb3ac59eb118 (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
I20260923 17:31:17.829882 17509 raft_consensus.cc:482] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.834952 17509 raft_consensus.cc:504] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.837101 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:17.855228 17143 tablet_bootstrap.cc:492] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:17.855685 17143 ts_tablet_manager.cc:1397] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.554s user 0.010s sys 0.001s
I20260923 17:31:17.857025 17146 tablet_bootstrap.cc:654] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:17.859082 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:17.866415 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_95f3f4adfca44bc791a565a2d16257f4 [id=2e8bb6c24f454c97882afd519641822f]). Expected version 1 got 0
I20260923 17:31:17.867900 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:17.868842 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:17.869455 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
W20260923 17:31:17.877799 16972 leader_election.cc:341] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.865909 17509 leader_election.cc:288] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:17.882201 16827 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:17.882609 16827 leader_election.cc:302] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:17.889237 17332 raft_consensus.cc:2726] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:17.894163 17143 raft_consensus.cc:348] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.895167 17143 raft_consensus.cc:374] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.896098 17143 raft_consensus.cc:729] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.898233 17143 consensus_queue.cc:260] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.896587 17509 raft_consensus.cc:482] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:17.903296 17509 raft_consensus.cc:504] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.904803 17146 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:17.905210 17146 ts_tablet_manager.cc:1397] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.098s user 0.009s sys 0.012s
I20260923 17:31:17.908629 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ccd82f191942239ffb88459d362107" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:17.915050 16972 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:17.905210 17509 leader_election.cc:288] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:17.923964 17146 raft_consensus.cc:348] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.930109 16379 catalog_manager.cc:5495] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
W20260923 17:31:17.932322 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fba670ca9ed0470e95dc7eb77bca73e4 consensus state has no leader
I20260923 17:31:17.933393 17146 raft_consensus.cc:374] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:17.933643 17146 raft_consensus.cc:729] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:17.933708 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_cfb90e755e9d41a5bf5d6636cecfb5a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:17.935151 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfb90e755e9d41a5bf5d6636cecfb5a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:17.938128 17146 consensus_queue.cc:260] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:17.950460 17146 ts_tablet_manager.cc:1428] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.045s user 0.007s sys 0.000s
I20260923 17:31:17.968042 17146 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:17.829342 16678 leader_election.cc:302] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:17.980397 17430 raft_consensus.cc:2781] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
W20260923 17:31:17.984294 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 155053us
I20260923 17:31:17.989221 17143 ts_tablet_manager.cc:1428] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.133s user 0.007s sys 0.000s
I20260923 17:31:18.006546 17146 tablet_bootstrap.cc:654] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.019629 17405 consensus_queue.cc:1035] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:18.031219 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.033227 17144 tablet_bootstrap.cc:654] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.032866 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ccd82f191942239ffb88459d362107" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:18.034859 16972 leader_election.cc:341] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.035238 16972 leader_election.cc:302] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.035881 17509 raft_consensus.cc:2726] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:18.036535 16972 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.036890 16972 leader_election.cc:302] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.037861 17509 raft_consensus.cc:2726] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:18.046051 17430 raft_consensus.cc:686] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:18.046942 17430 consensus_queue.cc:237] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.056497 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 99) took 1606 ms. Trace:
I20260923 17:31:18.056862 17403 rpcz_store.cc:276] 0923 17:31:16.449731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.450120 (+ 389us) service_pool.cc:227] Handling call
0923 17:31:17.714016 (+1263896us) ts_tablet_manager.cc:1145] Deleting tablet 8879dee684214c6dac3f9d6642a6d97d
0923 17:31:17.714072 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:17.771342 (+ 57270us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:17.797447 (+ 26105us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.056432 (+258985us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18857,"tablet-delete.queue_time_us":1221600}
I20260923 17:31:18.059710 17332 raft_consensus.cc:482] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.060391 17332 raft_consensus.cc:504] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.061836 17146 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:18.062237 17146 ts_tablet_manager.cc:1397] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.098s user 0.009s sys 0.001s
I20260923 17:31:18.062450 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 8879dee684214c6dac3f9d6642a6d97d (table default.Table_d7bb20f8a5544384ab1e69311e05400e [id=0392d68a37394099a98ab6f6e529c118]) successfully deleted
I20260923 17:31:18.066741 17285 consensus_queue.cc:1035] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:18.072580 16385 catalog_manager.cc:5495] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:18.075413 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:18.076921 17146 raft_consensus.cc:348] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.077574 17146 raft_consensus.cc:374] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.077798 17146 raft_consensus.cc:729] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.084038 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 100) took 1634 ms. Trace:
I20260923 17:31:18.084399 17403 rpcz_store.cc:276] 0923 17:31:16.450024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.450807 (+ 783us) service_pool.cc:227] Handling call
0923 17:31:18.057874 (+1607067us) ts_tablet_manager.cc:1145] Deleting tablet f124169d87c240a5bd544d3b6693323f
0923 17:31:18.083798 (+ 25924us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:18.083981 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1606550}
W20260923 17:31:18.086006 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet f124169d87c240a5bd544d3b6693323f because the tablet was not found. No further retry: Not found: Tablet not found: f124169d87c240a5bd544d3b6693323f
I20260923 17:31:18.086072 17146 consensus_queue.cc:260] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.087231 17403 tablet_replica.cc:330] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:18.088506 17403 raft_consensus.cc:2229] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.090443 17403 raft_consensus.cc:2258] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.093876 17146 ts_tablet_manager.cc:1428] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260923 17:31:18.094867 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:18.095511 17041 raft_consensus.cc:2454] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:18.097321 17403 ts_tablet_manager.cc:1905] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.098263 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:18.098387 16827 leader_election.cc:302] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:18.099546 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:18.099722 17405 raft_consensus.cc:2781] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:18.100016 17405 raft_consensus.cc:482] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.100330 17405 raft_consensus.cc:3037] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.103547 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:18.105126 17332 leader_election.cc:288] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.106534 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.107828 16386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_af6ea13bdd26434f9222c9255b68ee1e [id=98651426b57f456a9a13bd9bc11a27f8] alter complete (version 2)
I20260923 17:31:18.114518 17405 raft_consensus.cc:504] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.119902 17405 leader_election.cc:288] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.121626 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:18.121990 17013 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.122226 17040 raft_consensus.cc:3037] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.122570 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.123225 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:18.125272 16374 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet fba670ca9ed0470e95dc7eb77bca73e4 consensus state has no leader
I20260923 17:31:18.126891 16381 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 145) took 2477 ms. Trace:
I20260923 17:31:18.129201 16383 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 150) took 1549 ms. Trace:
I20260923 17:31:18.133962 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.134743 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.139654 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.142062 17143 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:18.143468 16387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 23) took 2031 ms (client timeout 1000 ms). Trace:
I20260923 17:31:18.145458 17146 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:18.145598 16826 consensus_peers.cc:487] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:18.147428 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.148010 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.148694 16744 raft_consensus.cc:2454] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:18.149966 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a182b6ced72428e9638465f0cc4e7ed" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:18.150473 16744 raft_consensus.cc:3037] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.158381 16744 raft_consensus.cc:2454] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:18.159508 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.159968 17343 tablet_replica.cc:330] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:18.160735 17343 raft_consensus.cc:2229] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.161175 17343 raft_consensus.cc:2258] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.161208 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.162578 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.172461 17509 raft_consensus.cc:482] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.172953 17509 raft_consensus.cc:504] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.174952 17509 leader_election.cc:288] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:18.176339 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051f2e478a324df98647d90107d55533" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:18.178012 16972 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.181521 17143 tablet_bootstrap.cc:654] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.181625 17509 raft_consensus.cc:482] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.182265 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051f2e478a324df98647d90107d55533" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.182052 17509 raft_consensus.cc:504] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:18.183604 16972 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.181958 17343 ts_tablet_manager.cc:1905] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:18.190544 16387 rpcz_store.cc:269] 0923 17:31:16.112434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.222368 (+109934us) service_pool.cc:227] Handling call
0923 17:31:16.527919 (+305551us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:18.143393 (+1615474us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":50869,"reported_tablets":18,"spinlock_wait_cycles":619110784}
I20260923 17:31:18.183951 16972 leader_election.cc:302] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.127202 16381 rpcz_store.cc:276] 0923 17:31:15.649329 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.320526 (+671197us) service_pool.cc:227] Handling call
0923 17:31:16.507244 (+186718us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df466 00007f9127120b34 00007f9127120ac2 00007f91271209f1 00007f912712096c 00007f912711fc22 00007f91253be94c 00007f91253bdd86
0923 17:31:16.508306 (+ 1062us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:16.508768 (+ 462us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:16.508894 (+ 126us) catalog_manager.cc:2761] Locking tablets
0923 17:31:17.200289 (+691395us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:17.250149 (+ 49860us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:17.276283 (+ 26134us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:18.099452 (+823169us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:18.126823 (+ 27371us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:17.203384 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:17.203905 (+ 521us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:17.203934 (+ 29us) write_op.cc:435] Acquired schema lock
0923 17:31:17.203947 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:17.204352 (+ 405us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:17.204401 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:17.204423 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:17.204722 (+ 299us) tablet.cc:699] Row locks acquired
0923 17:31:17.204737 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:17.204874 (+ 137us) write_op.cc:270] Start()
0923 17:31:17.204968 (+ 94us) write_op.cc:276] Timestamp: P: 1790184677204702 usec, L: 0
0923 17:31:17.204984 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:17.205555 (+ 571us) log.cc:842] Serialized 768 byte log entry
0923 17:31:17.244797 (+ 39242us) op_driver.cc:444] REPLICATION: finished
0923 17:31:17.248106 (+ 3309us) write_op.cc:301] APPLY: starting
0923 17:31:17.248173 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:17.248316 (+ 143us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:17.248429 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:17.248696 (+ 267us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:17.248867 (+ 171us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:17.248914 (+ 47us) write_op.cc:312] APPLY: finished
0923 17:31:17.249140 (+ 226us) log.cc:842] Serialized 29 byte log entry
0923 17:31:17.249829 (+ 689us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.249907 (+ 78us) write_op.cc:454] Released schema lock
0923 17:31:17.249945 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:17.250130 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.250153 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":427299200,"child_traces":[["op",{"apply.queue_time_us":3120,"apply.run_cpu_time_us":1746,"apply.run_wall_time_us":2256,"num_ops":2,"prepare.queue_time_us":1220,"prepare.run_cpu_time_us":2583,"prepare.run_wall_time_us":4466,"raft.queue_time_us":172,"raft.run_cpu_time_us":673,"raft.run_wall_time_us":676,"replication_time_us":39714}]]}
I20260923 17:31:18.194185 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_7e13d4e4841d47c18ecc2442c83ec934"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:18.195779 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e13d4e4841d47c18ecc2442c83ec934 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:18.165069 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because the tablet was not found. No further retry: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:18.196115 17144 tablet_bootstrap.cc:492] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:18.196632 17144 ts_tablet_manager.cc:1397] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.396s user 0.007s sys 0.017s
W20260923 17:31:18.196698 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 723000dc175844e990740dc503b53ced because the tablet was not found. No further retry: Not found: Tablet not found: 723000dc175844e990740dc503b53ced
W20260923 17:31:18.197285 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a3bcaddfad4f45739a705c17bdfe332f because the tablet was not found. No further retry: Not found: Tablet not found: a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:18.129483 16383 rpcz_store.cc:276] 0923 17:31:16.579521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.624789 (+ 45268us) service_pool.cc:227] Handling call
0923 17:31:16.626072 (+ 1283us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:16.626598 (+ 526us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:16.626718 (+ 120us) catalog_manager.cc:2761] Locking tablets
0923 17:31:16.627238 (+ 520us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:16.825220 (+197982us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:16.825612 (+ 392us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:18.103445 (+1277833us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:18.129140 (+ 25695us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.773666 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.774041 (+ 375us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.774066 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:16.774092 (+ 26us) tablet.cc:660] Decoding operations
0923 17:31:16.774412 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.774458 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.774481 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:16.774761 (+ 280us) tablet.cc:699] Row locks acquired
0923 17:31:16.774776 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:16.774914 (+ 138us) write_op.cc:270] Start()
0923 17:31:16.775003 (+ 89us) write_op.cc:276] Timestamp: P: 1790184676774863 usec, L: 0
0923 17:31:16.775019 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.775555 (+ 536us) log.cc:842] Serialized 768 byte log entry
0923 17:31:16.784192 (+ 8637us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.784605 (+ 413us) write_op.cc:301] APPLY: starting
0923 17:31:16.784656 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.784754 (+ 98us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.784853 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.785066 (+ 213us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.785195 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:16.785234 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:16.785399 (+ 165us) log.cc:842] Serialized 29 byte log entry
0923 17:31:16.785531 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.785601 (+ 70us) write_op.cc:454] Released schema lock
0923 17:31:16.785631 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.785795 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.785817 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":244,"apply.run_cpu_time_us":1377,"apply.run_wall_time_us":1376,"num_ops":2,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":2228,"prepare.run_wall_time_us":2228,"raft.queue_time_us":2597,"raft.run_cpu_time_us":628,"raft.run_wall_time_us":629,"replication_time_us":9099}]]}
I20260923 17:31:18.198530 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet f124169d87c240a5bd544d3b6693323f with delete_type TABLET_DATA_DELETED (replaced by tablet e92cfe501505458ea46da095a8c0d30a) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.198513 17430 consensus_queue.cc:1035] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:18.199759 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c1fb4b15a00641eda2af13db76fd5ec6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:18.201656 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1fb4b15a00641eda2af13db76fd5ec6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:18.208206 17144 raft_consensus.cc:348] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.208871 17144 raft_consensus.cc:374] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.209113 17144 raft_consensus.cc:729] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.209764 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.209942 17144 consensus_queue.cc:260] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.222067 17430 consensus_queue.cc:1035] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:18.225405 17144 ts_tablet_manager.cc:1428] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.029s user 0.004s sys 0.001s
W20260923 17:31:18.189621 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 22) took 3060 ms (client timeout 1000 ms). Trace:
W20260923 17:31:18.234670 16384 rpcz_store.cc:269] 0923 17:31:15.129559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:15.202915 (+ 73356us) service_pool.cc:227] Handling call
0923 17:31:15.218592 (+ 15677us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:18.189542 (+2970950us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":124814,"reported_tablets":18,"spinlock_wait_cycles":25852800}
I20260923 17:31:18.236670 16384 master_service.cc:946] ReplaceTablet: received request to replace tablet 21a629ce6ab14d12b6a83ce6bc56f70f from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:18.183984 17509 leader_election.cc:288] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:18.243393 17343 ts_tablet_manager.cc:1918] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:18.246778 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.206984 16826 leader_election.cc:302] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
W20260923 17:31:18.251608 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 44669.4us
I20260923 17:31:18.251816 17332 raft_consensus.cc:2781] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:18.252729 17332 raft_consensus.cc:686] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:18.209239 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.213503 17536 raft_consensus.cc:2726] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:18.265475 17541 tablet.cc:1722] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:18.267863 17144 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:18.250665 17040 raft_consensus.cc:2454] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:18.281363 17343 log.cc:1196] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:18.283205 16381 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 159) took 1015 ms. Trace:
I20260923 17:31:18.285087 17343 ts_tablet_manager.cc:1939] T d22f0f9d7e6a46a08ca9f411cc2c658b P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:18.287454 17343 tablet_replica.cc:330] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:18.292882 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet d22f0f9d7e6a46a08ca9f411cc2c658b (table default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]) successfully deleted
I20260923 17:31:18.290304 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.302547 16383 master_service.cc:946] ReplaceTablet: received request to replace tablet 051f2e478a324df98647d90107d55533 from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:18.306211 16678 consensus_peers.cc:487] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:18.293449 17332 consensus_queue.cc:237] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.184020 17146 tablet_bootstrap.cc:654] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.288136 17343 raft_consensus.cc:2229] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.311635 17343 raft_consensus.cc:2258] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 17:31:18.319007 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 135431us
I20260923 17:31:18.320688 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:18.322392 16972 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.325392 17403 ts_tablet_manager.cc:1918] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:18.325766 17403 log.cc:1196] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/1483835b2bc64873a3570ad17a079713
I20260923 17:31:18.326388 17403 ts_tablet_manager.cc:1939] T 1483835b2bc64873a3570ad17a079713 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:18.327123 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 101) took 1874 ms. Trace:
I20260923 17:31:18.327518 17403 rpcz_store.cc:276] 0923 17:31:16.452299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.452470 (+ 171us) service_pool.cc:227] Handling call
0923 17:31:18.087072 (+1634602us) ts_tablet_manager.cc:1145] Deleting tablet 1483835b2bc64873a3570ad17a079713
0923 17:31:18.087113 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:18.194127 (+107014us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.325145 (+131018us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.327073 (+ 1928us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1634101}
I20260923 17:31:18.283704 16381 rpcz_store.cc:276] 0923 17:31:17.267872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.193712 (+925840us) service_pool.cc:227] Handling call
0923 17:31:18.196163 (+ 2451us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:18.229530 (+ 33367us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:18.282341 (+ 52811us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.282369 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:18.282574 (+ 205us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:18.282869 (+ 295us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:18.283136 (+ 267us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:18.230772 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:18.231294 (+ 522us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:18.231320 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:18.231333 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:18.231637 (+ 304us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:18.231680 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:18.231702 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:18.231897 (+ 195us) tablet.cc:699] Row locks acquired
0923 17:31:18.231912 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:18.232031 (+ 119us) write_op.cc:270] Start()
0923 17:31:18.232128 (+ 97us) write_op.cc:276] Timestamp: P: 1790184678231990 usec, L: 0
0923 17:31:18.232145 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:18.232634 (+ 489us) log.cc:842] Serialized 433 byte log entry
0923 17:31:18.266781 (+ 34147us) op_driver.cc:444] REPLICATION: finished
0923 17:31:18.267123 (+ 342us) write_op.cc:301] APPLY: starting
0923 17:31:18.267177 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:18.267280 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:18.267396 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:18.267793 (+ 397us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:18.267930 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:18.267971 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:18.268177 (+ 206us) log.cc:842] Serialized 29 byte log entry
0923 17:31:18.268353 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.268431 (+ 78us) write_op.cc:454] Released schema lock
0923 17:31:18.268463 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":172,"apply.run_cpu_time_us":1538,"apply.run_wall_time_us":1537,"num_ops":2,"prepare.queue_time_us":86,"prepare.run_cpu_time_us":1974,"prepare.run_wall_time_us":2117,"raft.queue_time_us":149,"raft.run_cpu_time_us":580,"raft.run_wall_time_us":1411,"replication_time_us":34544}]]}
I20260923 17:31:18.330921 17343 ts_tablet_manager.cc:1905] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.345377 17537 raft_consensus.cc:482] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.345990 17537 raft_consensus.cc:504] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.348825 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.351734 17537 leader_election.cc:288] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:18.354247 17540 tablet.cc:1722] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:18.360886 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_5b64b645bddd49ec8eb278fa41130724" } new_table_name: "default.Table_80b04aa8dd0c4d07a921ae68656507c2" modify_external_catalogs: true
W20260923 17:31:18.364820 16972 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.333647 17144 tablet_bootstrap.cc:654] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.353260 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.365367 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 1483835b2bc64873a3570ad17a079713 (table default.Table_b8377c11b1cc4acebc14e04381f6c57b [id=27b02ea6c00e4bf6b183d3f2ff12affa]) successfully deleted
I20260923 17:31:18.366446 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 102) took 1912 ms. Trace:
I20260923 17:31:18.366904 17403 rpcz_store.cc:276] 0923 17:31:16.454290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.454383 (+ 93us) service_pool.cc:227] Handling call
0923 17:31:18.366202 (+1911819us) ts_tablet_manager.cc:1145] Deleting tablet 21fc7428f55041b6b939a8ca08574b90
0923 17:31:18.366243 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:18.366385 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911295}
W20260923 17:31:18.368156 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 21fc7428f55041b6b939a8ca08574b90 because the tablet was not found. No further retry: Not found: Tablet not found: 21fc7428f55041b6b939a8ca08574b90
I20260923 17:31:18.369046 17403 tablet_replica.cc:330] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:18.369714 16972 leader_election.cc:302] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.369807 17403 raft_consensus.cc:2229] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.370566 17536 raft_consensus.cc:2726] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:18.370911 16972 leader_election.cc:341] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.368782 17537 raft_consensus.cc:482] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.372351 17537 raft_consensus.cc:504] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.373850 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.376880 17542 tablet_replica.cc:330] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:18.378342 17542 raft_consensus.cc:2229] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.378821 17542 raft_consensus.cc:2258] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.389501 17146 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:18.389979 17146 ts_tablet_manager.cc:1397] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.245s user 0.012s sys 0.007s
I20260923 17:31:18.390646 17537 leader_election.cc:288] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:18.394793 17403 raft_consensus.cc:2258] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.395083 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ccdf1bd7a64f51965e0a6dc7e033c9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.397218 17403 ts_tablet_manager.cc:1905] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.396152 17143 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:18.403544 17143 ts_tablet_manager.cc:1397] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.262s user 0.027s sys 0.000s
W20260923 17:31:18.405750 16972 leader_election.cc:341] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.406204 16972 leader_election.cc:302] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:18.407250 16972 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:18.408054 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a90f7348a964a7f871e93dde7ab82f2 has no consensus state
I20260923 17:31:18.409237 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b4fa8697648e41faba9af136a0f73a3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:18.412169 17537 raft_consensus.cc:2726] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:18.411100 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4fa8697648e41faba9af136a0f73a3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:18.413245 16373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260923 17:31:18.419354 17542 ts_tablet_manager.cc:1905] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.420352 17143 raft_consensus.cc:348] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.421036 17143 raft_consensus.cc:374] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.421275 17143 raft_consensus.cc:729] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.421809 17143 consensus_queue.cc:260] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.431851 17143 ts_tablet_manager.cc:1428] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260923 17:31:18.432693 17143 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:18.392127 17146 raft_consensus.cc:348] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.433732 17146 raft_consensus.cc:374] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.433983 17146 raft_consensus.cc:729] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.434566 17146 consensus_queue.cc:260] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.436388 17146 ts_tablet_manager.cc:1428] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.046s user 0.005s sys 0.000s
I20260923 17:31:18.452088 17143 tablet_bootstrap.cc:654] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.463554 17146 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:18.469470 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ccdf1bd7a64f51965e0a6dc7e033c9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.470175 17144 tablet_bootstrap.cc:492] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:18.470532 17144 ts_tablet_manager.cc:1397] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.203s user 0.011s sys 0.006s
W20260923 17:31:18.473001 16972 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.473417 16972 leader_election.cc:302] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.474030 17537 raft_consensus.cc:2726] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:18.478900 17144 raft_consensus.cc:348] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.479499 17144 raft_consensus.cc:374] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.479658 17343 ts_tablet_manager.cc:1918] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:18.479717 17144 raft_consensus.cc:729] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.479972 17343 log.cc:1196] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:18.483875 17144 consensus_queue.cc:260] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.487991 17343 ts_tablet_manager.cc:1939] T a97e944f4e0c48db813e3bee61102c53 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:18.488265 17429 raft_consensus.cc:482] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.488683 17429 raft_consensus.cc:504] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.490411 17143 tablet_bootstrap.cc:492] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:18.490422 17429 leader_election.cc:288] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.490851 17143 ts_tablet_manager.cc:1397] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.058s user 0.010s sys 0.002s
I20260923 17:31:18.490965 17537 raft_consensus.cc:482] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.491325 17537 raft_consensus.cc:504] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.494903 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2beb8360d54efeb2fa74c005d011d1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.495115 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.495532 16893 raft_consensus.cc:2454] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:18.495631 16678 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
W20260923 17:31:18.496357 16972 leader_election.cc:341] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:18.491544 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 78308.6us
I20260923 17:31:18.496286 17143 raft_consensus.cc:348] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:18.498420 16678 consensus_peers.cc:487] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:18.498647 17143 raft_consensus.cc:374] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.498901 17143 raft_consensus.cc:729] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.499353 16677 leader_election.cc:302] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:18.500624 17542 ts_tablet_manager.cc:1918] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:18.500530 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.500900 17528 raft_consensus.cc:2781] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:18.501016 17542 log.cc:1196] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:18.501181 17528 raft_consensus.cc:482] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.501473 17528 raft_consensus.cc:3037] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.501876 17542 ts_tablet_manager.cc:1939] T d22f0f9d7e6a46a08ca9f411cc2c658b P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
W20260923 17:31:18.503737 16972 leader_election.cc:341] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.504096 16972 leader_election.cc:302] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.504693 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet d22f0f9d7e6a46a08ca9f411cc2c658b (table default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]) successfully deleted
I20260923 17:31:18.504755 17536 raft_consensus.cc:2726] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:18.505451 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ce78da2e733545078e9762632b134528" } new_table_name: "default.Table_6ff2f38012514a27ba6260d78aab79eb" modify_external_catalogs: true
I20260923 17:31:18.505882 17537 leader_election.cc:288] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:18.507656 16678 consensus_peers.cc:487] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:18.510397 17143 consensus_queue.cc:260] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.518390 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a97e944f4e0c48db813e3bee61102c53 (table default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]) successfully deleted
W20260923 17:31:18.521008 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 2c842eff86634bf4bc70c426b5226d66 because the tablet was not found. No further retry: Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:18.522598 17343 tablet_replica.cc:330] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:18.524161 17144 ts_tablet_manager.cc:1428] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.053s user 0.006s sys 0.001s
I20260923 17:31:18.524470 17143 ts_tablet_manager.cc:1428] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260923 17:31:18.525427 17144 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:18.524310 17343 raft_consensus.cc:2229] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.527486 17343 raft_consensus.cc:2258] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.529968 17343 ts_tablet_manager.cc:1905] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.531215 17528 raft_consensus.cc:504] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.532400 17143 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:18.534724 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2beb8360d54efeb2fa74c005d011d1" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:18.535434 16893 raft_consensus.cc:3037] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:18.535877 16678 leader_election.cc:341] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:18.537181 17542 tablet_replica.cc:330] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:18.538031 17542 raft_consensus.cc:2229] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:18.538983 17542 raft_consensus.cc:2258] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.540992 17542 ts_tablet_manager.cc:1905] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.546223 17144 tablet_bootstrap.cc:654] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.562250 17143 tablet_bootstrap.cc:654] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.570338 17285 raft_consensus.cc:482] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.570834 17285 raft_consensus.cc:504] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.572938 17285 leader_election.cc:288] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.573287 17528 leader_election.cc:288] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.468150 17146 tablet_bootstrap.cc:654] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.575542 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:18.576149 17040 raft_consensus.cc:2454] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:18.576635 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.577193 16888 raft_consensus.cc:2454] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:18.577318 16678 leader_election.cc:302] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:18.581687 16893 raft_consensus.cc:2454] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:18.583096 16677 leader_election.cc:302] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:18.584323 17430 raft_consensus.cc:2781] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:18.584515 17285 raft_consensus.cc:2781] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:18.584702 17430 raft_consensus.cc:482] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.585054 17430 raft_consensus.cc:3037] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.585992 17285 raft_consensus.cc:686] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:18.586705 17285 consensus_queue.cc:237] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.591902 17550 consensus_queue.cc:1035] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:18.598538 17430 raft_consensus.cc:504] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.598990 17473 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.599525 17473 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.600339 17343 ts_tablet_manager.cc:1918] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:18.600670 17343 log.cc:1196] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:18.601239 17343 ts_tablet_manager.cc:1939] T 1a182b6ced72428e9638465f0cc4e7ed P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:18.601352 17473 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.603058 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.603717 16888 raft_consensus.cc:2454] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:18.603974 17430 leader_election.cc:288] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.606204 17143 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:18.605628 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:18.606761 17143 ts_tablet_manager.cc:1397] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.080s user 0.006s sys 0.005s
I20260923 17:31:18.607023 16888 raft_consensus.cc:3037] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.608851 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2796fcb72f5497a92208b40224739e9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:18.609525 17040 raft_consensus.cc:3037] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.612874 17143 raft_consensus.cc:348] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.615468 17143 raft_consensus.cc:374] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.615753 17143 raft_consensus.cc:729] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.616891 17040 raft_consensus.cc:2454] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:18.617442 17143 consensus_queue.cc:260] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.613551 16677 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:18.618546 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 1a182b6ced72428e9638465f0cc4e7ed (table default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]) successfully deleted
W20260923 17:31:18.618695 16678 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:18.619378 17528 raft_consensus.cc:2781] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 17:31:18.619704 17528 raft_consensus.cc:482] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.620007 17528 raft_consensus.cc:3037] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Advancing to term 2
W20260923 17:31:18.621125 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because the tablet was not found. No further retry: Not found: Tablet not found: 8879dee684214c6dac3f9d6642a6d97d
W20260923 17:31:18.622825 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 because the tablet was not found. No further retry: Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1
W20260923 17:31:18.625013 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because the tablet was not found. No further retry: Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
W20260923 17:31:18.627169 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f124169d87c240a5bd544d3b6693323f because the tablet was not found. No further retry: Not found: Tablet not found: f124169d87c240a5bd544d3b6693323f
I20260923 17:31:18.629415 17542 ts_tablet_manager.cc:1918] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:18.629854 17542 log.cc:1196] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a97e944f4e0c48db813e3bee61102c53
W20260923 17:31:18.630187 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 0721fa178c0347598351b9d3512089b8 because the tablet was not found. No further retry: Not found: Tablet not found: 0721fa178c0347598351b9d3512089b8
I20260923 17:31:18.630587 17542 ts_tablet_manager.cc:1939] T a97e944f4e0c48db813e3bee61102c53 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:18.642711 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a97e944f4e0c48db813e3bee61102c53 (table default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]) successfully deleted
I20260923 17:31:18.646267 16678 leader_election.cc:302] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:18.647023 17430 raft_consensus.cc:2781] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:18.648031 17430 raft_consensus.cc:686] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:18.648893 17430 consensus_queue.cc:237] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.655452 16888 raft_consensus.cc:2454] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:18.638428 17146 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:18.656401 17146 ts_tablet_manager.cc:1397] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.193s user 0.009s sys 0.000s
I20260923 17:31:18.658753 17146 raft_consensus.cc:348] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.661252 17550 raft_consensus.cc:482] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.661762 17550 raft_consensus.cc:504] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.663228 17528 raft_consensus.cc:504] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.663897 17550 leader_election.cc:288] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.668130 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.670027 17144 tablet_bootstrap.cc:492] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:18.670428 17144 ts_tablet_manager.cc:1397] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.145s user 0.008s sys 0.000s
W20260923 17:31:18.671198 16826 leader_election.cc:341] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.672775 17144 raft_consensus.cc:348] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.675513 17144 raft_consensus.cc:374] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.675784 17144 raft_consensus.cc:729] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.677691 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 051f2e478a324df98647d90107d55533
I20260923 17:31:18.679129 16383 catalog_manager.cc:6565] ReplaceTablet: tablet 051f2e478a324df98647d90107d55533 deleted and replaced by tablet 107a073567314be996d6b8e3dbc47701
I20260923 17:31:18.679461 16383 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 162) took 1060 ms. Trace:
I20260923 17:31:18.680764 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.678546 17144 consensus_queue.cc:260] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.679738 16383 rpcz_store.cc:276] 0923 17:31:17.619237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.302357 (+683120us) service_pool.cc:227] Handling call
0923 17:31:18.679402 (+377045us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:18.662694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:18.663064 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:18.663088 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:18.663099 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:18.663412 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:18.663455 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:18.663477 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:18.663702 (+ 225us) tablet.cc:699] Row locks acquired
0923 17:31:18.663716 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:18.663835 (+ 119us) write_op.cc:270] Start()
0923 17:31:18.663928 (+ 93us) write_op.cc:276] Timestamp: P: 1790184678663764 usec, L: 0
0923 17:31:18.663945 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:18.664806 (+ 861us) log.cc:842] Serialized 631 byte log entry
0923 17:31:18.673579 (+ 8773us) op_driver.cc:444] REPLICATION: finished
0923 17:31:18.673934 (+ 355us) write_op.cc:301] APPLY: starting
0923 17:31:18.673998 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:18.674123 (+ 125us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:18.674248 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:18.674715 (+ 467us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:18.674970 (+ 255us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:18.675016 (+ 46us) write_op.cc:312] APPLY: finished
0923 17:31:18.675236 (+ 220us) log.cc:842] Serialized 29 byte log entry
0923 17:31:18.675393 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.675462 (+ 69us) write_op.cc:454] Released schema lock
0923 17:31:18.675492 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:18.675686 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.675719 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":184,"apply.run_cpu_time_us":1970,"apply.run_wall_time_us":1969,"num_ops":2,"prepare.queue_time_us":277,"prepare.run_cpu_time_us":2220,"prepare.run_wall_time_us":3555,"raft.queue_time_us":187,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":1388,"replication_time_us":9536}]]}
I20260923 17:31:18.683457 17144 ts_tablet_manager.cc:1428] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260923 17:31:18.684849 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_841cfa422dfb457eb8f606f44dfa4c7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:18.685948 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 051f2e478a324df98647d90107d55533 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 051f2e478a324df98647d90107d55533 already in progress: creating tablet
W20260923 17:31:18.686432 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_841cfa422dfb457eb8f606f44dfa4c7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:18.686913 16827 consensus_peers.cc:487] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:18.688632 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.672477 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:18.690711 17144 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:18.680176 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.691625 17040 raft_consensus.cc:2454] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:18.692631 17146 raft_consensus.cc:374] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.692659 16827 leader_election.cc:302] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217
I20260923 17:31:18.692957 17146 raft_consensus.cc:729] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.693776 17550 raft_consensus.cc:2781] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:18.694110 17550 raft_consensus.cc:482] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.693735 17146 consensus_queue.cc:260] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.694392 17550 raft_consensus.cc:3037] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.690711 17403 ts_tablet_manager.cc:1918] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:18.700467 17403 log.cc:1196] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/21a629ce6ab14d12b6a83ce6bc56f70f
I20260923 17:31:18.700663 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 6eb728f7f83e4975823e338a4568f796 (DEFAULT_TABLE table=default.Table_bcf64eeacb1b4b85afb1ef76d08123f1 [id=e219e77c4bf94425aea79fefc278aa09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.700836 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d65cfb2eb3648faab5869bba69cab3f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:18.701109 17403 ts_tablet_manager.cc:1939] T 21a629ce6ab14d12b6a83ce6bc56f70f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:18.701411 16885 raft_consensus.cc:3037] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 17:31:18.701762 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eb728f7f83e4975823e338a4568f796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.702584 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 6eb728f7f83e4975823e338a4568f796 (DEFAULT_TABLE table=default.Table_bcf64eeacb1b4b85afb1ef76d08123f1 [id=e219e77c4bf94425aea79fefc278aa09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.702857 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 103) took 2247 ms. Trace:
I20260923 17:31:18.703117 17403 rpcz_store.cc:276] 0923 17:31:16.455252 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.455345 (+ 93us) service_pool.cc:227] Handling call
0923 17:31:18.368904 (+1913559us) ts_tablet_manager.cc:1145] Deleting tablet 21a629ce6ab14d12b6a83ce6bc56f70f
0923 17:31:18.368943 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:18.580622 (+211679us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.690501 (+109879us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.702807 (+ 12306us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1913033}
I20260923 17:31:18.707360 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 6a90f7348a964a7f871e93dde7ab82f2 (DEFAULT_TABLE table=default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.710311 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 6a90f7348a964a7f871e93dde7ab82f2 (DEFAULT_TABLE table=default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.711129 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 21a629ce6ab14d12b6a83ce6bc56f70f (table default.Table_c9193016785148b492ccdc68f136e29b [id=2590267adffb4a43b9ff788d35a38a93]) successfully deleted
I20260923 17:31:18.712457 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 6eb728f7f83e4975823e338a4568f796 (DEFAULT_TABLE table=default.Table_bcf64eeacb1b4b85afb1ef76d08123f1 [id=e219e77c4bf94425aea79fefc278aa09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.712749 16885 raft_consensus.cc:2454] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 2.
I20260923 17:31:18.713979 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a90f7348a964a7f871e93dde7ab82f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.714396 16677 leader_election.cc:302] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:18.715371 17430 raft_consensus.cc:2781] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 17:31:18.715869 17430 raft_consensus.cc:686] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:18.716513 16382 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:18.717139 17514 raft_consensus.cc:482] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.716936 17430 consensus_queue.cc:237] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.717988 16382 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:18.717576 17514 raft_consensus.cc:504] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.718590 16382 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:18.719853 17514 leader_election.cc:288] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.723155 17144 tablet_bootstrap.cc:654] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.728652 17146 ts_tablet_manager.cc:1428] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.072s user 0.005s sys 0.000s
I20260923 17:31:18.732547 17403 tablet_replica.cc:330] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:18.733932 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 6a90f7348a964a7f871e93dde7ab82f2 (DEFAULT_TABLE table=default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.735634 17550 raft_consensus.cc:504] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.739318 17550 leader_election.cc:288] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.748466 17528 leader_election.cc:288] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:18.732446 17285 consensus_queue.cc:1035] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260923 17:31:18.753666 17143 ts_tablet_manager.cc:1428] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.147s user 0.007s sys 0.000s
I20260923 17:31:18.760041 16743 raft_consensus.cc:3037] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.762697 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.764321 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85eba2aac8d84defa8ce4081d447bc63" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:18.765149 17041 raft_consensus.cc:2379] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:18.765245 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
W20260923 17:31:18.766263 16826 leader_election.cc:341] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.765822 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80034cbf6354e1bbe0a0220e43251c3" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:18.767227 17037 raft_consensus.cc:3037] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.767632 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eb728f7f83e4975823e338a4568f796. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:18.768620 16678 leader_election.cc:341] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:18.771658 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6eb728f7f83e4975823e338a4568f796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.773312 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a90f7348a964a7f871e93dde7ab82f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.776530 17343 tablet_replica.cc:330] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:18.777808 16827 leader_election.cc:302] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:18.777916 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:18.779032 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:18.779247 17514 raft_consensus.cc:3037] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.784862 17343 raft_consensus.cc:2229] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:18.785378 17343 raft_consensus.cc:2258] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.787103 17343 ts_tablet_manager.cc:1905] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.793025 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 1822edfa26b045d7a15074a4bd5cbf5e (DEFAULT_TABLE table=default.Table_363f829391944ba18bdfc4f2e1f34dce [id=6390bb4ab7c04d16ace74b9aca902753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.794171 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1822edfa26b045d7a15074a4bd5cbf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.794453 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 1822edfa26b045d7a15074a4bd5cbf5e (DEFAULT_TABLE table=default.Table_363f829391944ba18bdfc4f2e1f34dce [id=6390bb4ab7c04d16ace74b9aca902753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.795436 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1822edfa26b045d7a15074a4bd5cbf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.797197 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 1822edfa26b045d7a15074a4bd5cbf5e (DEFAULT_TABLE table=default.Table_363f829391944ba18bdfc4f2e1f34dce [id=6390bb4ab7c04d16ace74b9aca902753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.798183 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1822edfa26b045d7a15074a4bd5cbf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.802676 17146 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:18.807366 17144 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:18.807756 17144 ts_tablet_manager.cc:1397] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.124s user 0.006s sys 0.010s
I20260923 17:31:18.808763 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a90f7348a964a7f871e93dde7ab82f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.824476 16383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_40b23e3d14dd4c1da5acb4c38e0e466f" } modify_external_catalogs: true
I20260923 17:31:18.825217 16383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_40b23e3d14dd4c1da5acb4c38e0e466f" } modify_external_catalogs: true
I20260923 17:31:18.817138 17144 raft_consensus.cc:348] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.829339 17144 raft_consensus.cc:374] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.829648 17144 raft_consensus.cc:729] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.832970 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:18.834055 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:18.835474 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:18.836359 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:18.836958 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:18.838402 16384 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:18.839236 17037 raft_consensus.cc:2454] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:18.844472 17403 raft_consensus.cc:2229] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 LEADER]: Raft consensus shutting down.
I20260923 17:31:18.846608 17403 pending_rounds.cc:62] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:18.846798 17514 raft_consensus.cc:2726] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:18.846912 17403 pending_rounds.cc:68] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7332596444034674688 op_type: NO_OP noop_request { }
I20260923 17:31:18.847286 17403 raft_consensus.cc:2866] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:18.847602 17403 raft_consensus.cc:2258] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:18.849464 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 261d1d03290546969a74ca60beb890eb (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.850678 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261d1d03290546969a74ca60beb890eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.851686 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 261d1d03290546969a74ca60beb890eb (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.852821 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261d1d03290546969a74ca60beb890eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.854315 17144 consensus_queue.cc:260] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.856436 17144 ts_tablet_manager.cc:1428] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260923 17:31:18.857188 17144 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:18.875542 17143 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:18.885012 17565 tablet.cc:1722] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:18.892150 16871 tablet_service.cc:1467] Processing CreateTablet for tablet 47a0f16066164132a26927f5a830e677 (DEFAULT_TABLE table=default.Table_dc1abdd9ffc44b1d83f6e604bec652df [id=2a56d63c352c458e8d71db88933ee01c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.893208 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a0f16066164132a26927f5a830e677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:18.897356 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 47a0f16066164132a26927f5a830e677 (DEFAULT_TABLE table=default.Table_dc1abdd9ffc44b1d83f6e604bec652df [id=2a56d63c352c458e8d71db88933ee01c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.897927 17144 tablet_bootstrap.cc:654] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.898624 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a0f16066164132a26927f5a830e677. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:18.910007 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85eba2aac8d84defa8ce4081d447bc63 consensus state has no leader
W20260923 17:31:18.889586 17561 consensus_queue.cc:1159] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be93fda0843b44f5abc49e6f35c69217" status { last_received { term: 2 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 2 index: 1 } }
I20260923 17:31:18.849015 17537 raft_consensus.cc:482] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.912091 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2243b815d62f461ca66f77ac0701c362" } modify_external_catalogs: true
I20260923 17:31:18.912482 17403 ts_tablet_manager.cc:1905] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:18.912647 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2243b815d62f461ca66f77ac0701c362" } modify_external_catalogs: true
I20260923 17:31:18.912489 17537 raft_consensus.cc:504] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.914484 17537 leader_election.cc:288] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:18.873226 17146 tablet_bootstrap.cc:654] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:18.890456 17013 tablet_service.cc:1467] Processing CreateTablet for tablet 261d1d03290546969a74ca60beb890eb (DEFAULT_TABLE table=default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:18.915462 16380 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_fcfadb32897349d4962baa0dc45bd2da [id=6d073df311674a358a520dd1586a3345] alter complete (version 2)
I20260923 17:31:18.915723 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.916041 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261d1d03290546969a74ca60beb890eb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:18.917174 16972 leader_election.cc:341] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.896574 17536 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.919554 17536 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.919873 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:18.921152 16972 leader_election.cc:341] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.921476 17536 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:18.921494 16972 leader_election.cc:302] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.930904 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.931059 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.931985 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.932950 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.933706 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:18.934202 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because the tablet was not found. No further retry: Not found: Tablet not found: 8879dee684214c6dac3f9d6642a6d97d
W20260923 17:31:18.934775 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because the tablet was not found. No further retry: Not found: Tablet not found: 1483835b2bc64873a3570ad17a079713
I20260923 17:31:18.935551 17536 raft_consensus.cc:2726] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:18.936530 16972 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:18.937131 16972 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:18.937549 16972 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:18.938581 17536 raft_consensus.cc:2726] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:18.942168 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 30) took 3178 ms (client timeout 1000 ms). Trace:
I20260923 17:31:18.945371 17430 consensus_queue.cc:1035] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:18.947439 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.952761 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.953490 17542 tablet_replica.cc:330] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:18.954015 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.955039 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.955476 17403 ts_tablet_manager.cc:1918] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260923 17:31:18.955845 17403 log.cc:1196] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:18.961007 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet 1483835b2bc64873a3570ad17a079713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.960939 16827 leader_election.cc:302] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217
I20260923 17:31:18.961622 16723 tablet_service.cc:1514] Processing DeleteTablet for tablet 0721fa178c0347598351b9d3512089b8 with delete_type TABLET_DATA_DELETED (replaced by tablet db3504813e9c4698a546b6d6fa0fbcb6) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.962065 17514 raft_consensus.cc:2781] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:18.962154 16722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8879dee684214c6dac3f9d6642a6d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.962543 17514 raft_consensus.cc:686] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:18.962698 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a629ce6ab14d12b6a83ce6bc56f70f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:09 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.963208 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.963279 17514 consensus_queue.cc:237] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:18.963711 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 27) took 3029 ms (client timeout 1000 ms). Trace:
W20260923 17:31:18.967299 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 24) took 1834 ms (client timeout 1000 ms). Trace:
I20260923 17:31:18.969684 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.969789 17536 raft_consensus.cc:482] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:18.970180 17536 raft_consensus.cc:504] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.970607 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.973584 17403 ts_tablet_manager.cc:1939] T 8d65cfb2eb3648faab5869bba69cab3f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:18.973502 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:18.974000 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.974152 16885 raft_consensus.cc:2454] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:18.974882 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.975600 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 104) took 2519 ms. Trace:
I20260923 17:31:18.975519 16972 leader_election.cc:302] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:18.975781 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:18.976205 17537 raft_consensus.cc:2781] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:18.976574 17537 raft_consensus.cc:482] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:18.976862 17537 raft_consensus.cc:3037] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:18.979310 17144 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:18.979712 17144 ts_tablet_manager.cc:1397] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.123s user 0.007s sys 0.005s
I20260923 17:31:18.980880 17536 leader_election.cc:288] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:18.981784 17144 raft_consensus.cc:348] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.982399 17144 raft_consensus.cc:374] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:18.982617 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.982437 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:18.982682 17144 raft_consensus.cc:729] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:18.983117 16745 raft_consensus.cc:2454] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:18.983273 17144 consensus_queue.cc:260] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.983894 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:18.984184 17430 consensus_queue.cc:1035] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:18.985666 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.986640 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:18.987793 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d44ee57faf4a1d99e077fd781a519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:10 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:18.973201 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 31) took 2037 ms (client timeout 1000 ms). Trace:
W20260923 17:31:18.991786 16385 rpcz_store.cc:269] 0923 17:31:16.935495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:17.043254 (+107759us) service_pool.cc:227] Handling call
0923 17:31:18.973125 (+1929871us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:18.247505 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:18.247858 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:18.247881 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:18.247894 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:18.248166 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:18.248206 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:18.248255 (+ 49us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:18.248432 (+ 177us) tablet.cc:699] Row locks acquired
0923 17:31:18.248447 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:18.248576 (+ 129us) write_op.cc:270] Start()
0923 17:31:18.248661 (+ 85us) write_op.cc:276] Timestamp: P: 1790184678248531 usec, L: 0
0923 17:31:18.248678 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:18.249257 (+ 579us) log.cc:842] Serialized 536 byte log entry
0923 17:31:18.282778 (+ 33521us) op_driver.cc:444] REPLICATION: finished
0923 17:31:18.283075 (+ 297us) write_op.cc:301] APPLY: starting
0923 17:31:18.283129 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:18.283221 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:18.283322 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:18.283518 (+ 196us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:18.283651 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:18.283688 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:18.283881 (+ 193us) log.cc:842] Serialized 23 byte log entry
0923 17:31:18.284016 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.284081 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:18.284126 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:18.284314 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.284341 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29626,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":160,"apply.run_cpu_time_us":1409,"apply.run_wall_time_us":1407,"num_ops":1,"prepare.queue_time_us":8311,"prepare.run_cpu_time_us":1991,"prepare.run_wall_time_us":1991,"replication_time_us":34022}]]}
I20260923 17:31:18.994079 17537 raft_consensus.cc:504] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:18.996465 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_7bc6c27fedc843e68f19f6075fdb755b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:18.998282 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bc6c27fedc843e68f19f6075fdb755b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:19.002591 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.003902 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bcaddfad4f45739a705c17bdfe332f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:15 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.004911 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.005861 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.007315 17430 raft_consensus.cc:482] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.007771 17430 raft_consensus.cc:504] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.011207 17430 leader_election.cc:288] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.012756 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:19.014364 16677 leader_election.cc:341] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.014848 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.015615 17037 raft_consensus.cc:2454] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:18.984747 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 21a629ce6ab14d12b6a83ce6bc56f70f because the tablet was not found. No further retry: Not found: Tablet not found: 21a629ce6ab14d12b6a83ce6bc56f70f
W20260923 17:31:18.980160 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 28) took 1958 ms (client timeout 1000 ms). Trace:
I20260923 17:31:18.975904 17403 rpcz_store.cc:276] 0923 17:31:16.456572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.456686 (+ 114us) service_pool.cc:227] Handling call
0923 17:31:18.704321 (+2247635us) ts_tablet_manager.cc:1145] Deleting tablet 8d65cfb2eb3648faab5869bba69cab3f
0923 17:31:18.732255 (+ 27934us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:18.846240 (+113985us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54001306d4. stack: 00007f24a4f0c033 00007f24a4f0bf30 00007f24a4a8d534 00000000004db2b4 00000000004db249 00007f24abe18471 00007f24abe9b352 00007f24ac2b126c 00007f24acebecfa 00007f24aceeee10 00007f24aced5e0d 00007f24aced5db9 00007f24aced5d49 00007f24aced5d11 00007f24aced4f3d 0000000000580784
0923 17:31:18.924207 (+ 77967us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.955249 (+ 31042us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:18.975540 (+ 20291us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":255261440,"tablet-delete.queue_time_us":2247169}
W20260923 17:31:19.017833 16388 rpcz_store.cc:269] 0923 17:31:17.022010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:17.077452 (+ 55442us) service_pool.cc:227] Handling call
0923 17:31:18.980077 (+1902625us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:17.659952 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:17.660342 (+ 390us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:17.660364 (+ 22us) write_op.cc:435] Acquired schema lock
0923 17:31:17.660377 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:17.660670 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:17.660706 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:17.660726 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:17.660991 (+ 265us) tablet.cc:699] Row locks acquired
0923 17:31:17.661004 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:17.661112 (+ 108us) write_op.cc:270] Start()
0923 17:31:17.661192 (+ 80us) write_op.cc:276] Timestamp: P: 1790184677661086 usec, L: 0
0923 17:31:17.661207 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:17.661674 (+ 467us) log.cc:842] Serialized 919 byte log entry
0923 17:31:17.668284 (+ 6610us) op_driver.cc:444] REPLICATION: finished
0923 17:31:17.669041 (+ 757us) write_op.cc:301] APPLY: starting
0923 17:31:17.669095 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:17.669200 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:17.669295 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:17.669519 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:17.669644 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:17.669692 (+ 48us) write_op.cc:312] APPLY: finished
0923 17:31:17.669871 (+ 179us) log.cc:842] Serialized 29 byte log entry
0923 17:31:17.670009 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.670076 (+ 67us) write_op.cc:454] Released schema lock
0923 17:31:17.670110 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:17.670273 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:17.670293 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":616,"apply.run_cpu_time_us":1419,"apply.run_wall_time_us":1418,"mutex_wait_us":32671,"num_ops":2,"prepare.queue_time_us":32876,"prepare.run_cpu_time_us":2095,"prepare.run_wall_time_us":5674,"raft.queue_time_us":152,"raft.run_cpu_time_us":585,"raft.run_wall_time_us":586,"replication_time_us":6987}]]}
I20260923 17:31:19.021181 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 105) took 2563 ms. Trace:
I20260923 17:31:19.021467 17403 rpcz_store.cc:276] 0923 17:31:16.457571 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.457673 (+ 102us) service_pool.cc:227] Handling call
0923 17:31:19.020957 (+2563284us) ts_tablet_manager.cc:1145] Deleting tablet ceb787c0863847289400e62718d218a1
0923 17:31:19.020997 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:19.021134 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2562802}
I20260923 17:31:19.021976 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 8d65cfb2eb3648faab5869bba69cab3f (table default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]) successfully deleted
W20260923 17:31:19.022639 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet ceb787c0863847289400e62718d218a1 because the tablet was not found. No further retry: Not found: Tablet not found: ceb787c0863847289400e62718d218a1
I20260923 17:31:19.022686 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 108) took 1668 ms. Trace:
I20260923 17:31:19.022935 17403 rpcz_store.cc:276] 0923 17:31:17.354146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:17.360326 (+ 6180us) service_pool.cc:227] Handling call
0923 17:31:19.022403 (+1662077us) ts_tablet_manager.cc:1145] Deleting tablet a60bf118082b4e9991bfeb3ac59eb118
0923 17:31:19.022437 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:19.022642 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1661513}
I20260923 17:31:19.023656 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_1facf5dbeaf74ac2a94bf7f3d1aa7b1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:19.024894 17144 ts_tablet_manager.cc:1428] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.045s user 0.006s sys 0.001s
I20260923 17:31:19.026203 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:19.026788 16893 raft_consensus.cc:3037] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.026094 17144 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.033694 16893 raft_consensus.cc:2454] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:19.034924 16972 leader_election.cc:302] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:19.035807 17536 raft_consensus.cc:2781] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:19.039147 17542 raft_consensus.cc:2229] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:19.041231 17542 raft_consensus.cc:2258] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.048261 17403 tablet_replica.cc:330] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:19.049017 17403 raft_consensus.cc:2229] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:19.049856 17403 raft_consensus.cc:2258] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.051508 17403 ts_tablet_manager.cc:1905] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:19.059122 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6f62eadc6a94aedb4a8164c545cda17" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:19.059854 16745 raft_consensus.cc:3037] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.068457 16745 raft_consensus.cc:2454] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
W20260923 17:31:19.082743 16827 consensus_peers.cc:487] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:19.086045 17144 tablet_bootstrap.cc:654] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:19.100071 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1facf5dbeaf74ac2a94bf7f3d1aa7b1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:18.964031 16380 rpcz_store.cc:269] 0923 17:31:15.934491 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:16.129251 (+194760us) service_pool.cc:227] Handling call
0923 17:31:16.148745 (+ 19494us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:18.963631 (+2814886us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:18.151929 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:18.152344 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:18.152370 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:18.152383 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:18.152701 (+ 318us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:18.152743 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:18.152771 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:18.152928 (+ 157us) tablet.cc:699] Row locks acquired
0923 17:31:18.152941 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:18.153060 (+ 119us) write_op.cc:270] Start()
0923 17:31:18.153146 (+ 86us) write_op.cc:276] Timestamp: P: 1790184678153024 usec, L: 0
0923 17:31:18.153162 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:18.153670 (+ 508us) log.cc:842] Serialized 356 byte log entry
0923 17:31:18.177428 (+ 23758us) op_driver.cc:444] REPLICATION: finished
0923 17:31:18.177775 (+ 347us) write_op.cc:301] APPLY: starting
0923 17:31:18.177843 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:18.177945 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:18.178070 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:18.178293 (+ 223us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:18.179036 (+ 743us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:18.179076 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:18.179277 (+ 201us) log.cc:842] Serialized 23 byte log entry
0923 17:31:18.179437 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.179508 (+ 71us) write_op.cc:454] Released schema lock
0923 17:31:18.179538 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:18.179717 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.179739 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":8792832,"child_traces":[["op",{"apply.queue_time_us":176,"apply.run_cpu_time_us":2133,"apply.run_wall_time_us":2133,"num_ops":1,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":2090,"prepare.run_wall_time_us":16108,"raft.queue_time_us":14264,"raft.run_cpu_time_us":703,"raft.run_wall_time_us":1821,"replication_time_us":24170}]]}
I20260923 17:31:19.104382 16380 master_service.cc:946] ReplaceTablet: received request to replace tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:19.106196 17144 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
W20260923 17:31:18.988373 16379 rpcz_store.cc:269] 0923 17:31:17.133149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:17.370117 (+236968us) service_pool.cc:227] Handling call
0923 17:31:18.052310 (+682193us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.052338 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:18.967225 (+914887us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:17.995663 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:17.996092 (+ 429us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:17.996118 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:17.996132 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:17.996476 (+ 344us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:17.996523 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:17.996547 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:17.996723 (+ 176us) tablet.cc:699] Row locks acquired
0923 17:31:17.996738 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:17.996874 (+ 136us) write_op.cc:270] Start()
0923 17:31:17.996966 (+ 92us) write_op.cc:276] Timestamp: P: 1790184677996818 usec, L: 0
0923 17:31:17.996984 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:17.997583 (+ 599us) log.cc:842] Serialized 536 byte log entry
0923 17:31:18.005287 (+ 7704us) op_driver.cc:444] REPLICATION: finished
0923 17:31:18.005588 (+ 301us) write_op.cc:301] APPLY: starting
0923 17:31:18.005650 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:18.005738 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:18.005830 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:18.006022 (+ 192us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:18.006143 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:18.006182 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:18.006372 (+ 190us) log.cc:842] Serialized 23 byte log entry
0923 17:31:18.006529 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:18.006589 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:18.006618 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":11835,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":1228,"apply.run_wall_time_us":1230,"num_ops":1,"prepare.queue_time_us":62299,"prepare.run_cpu_time_us":2261,"prepare.run_wall_time_us":2262,"raft.queue_time_us":190,"raft.run_cpu_time_us":775,"raft.run_wall_time_us":773,"replication_time_us":8223}]]}
I20260923 17:31:19.106582 17144 ts_tablet_manager.cc:1397] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.081s user 0.005s sys 0.003s
I20260923 17:31:19.107439 16379 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3880da019d614925ba87aa4c506e1107" } new_table_name: "default.Table_b3879296b41f459ab8637f32909c42f6" modify_external_catalogs: true
I20260923 17:31:19.108721 17144 raft_consensus.cc:348] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.109182 17542 ts_tablet_manager.cc:1905] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:19.109355 17144 raft_consensus.cc:374] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.109597 17144 raft_consensus.cc:729] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.110373 17144 consensus_queue.cc:260] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.116624 17537 leader_election.cc:288] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:19.119097 17570 log.cc:924] Time spent T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
W20260923 17:31:18.948050 16386 rpcz_store.cc:269] 0923 17:31:15.763444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:15.884048 (+120604us) service_pool.cc:227] Handling call
0923 17:31:15.897530 (+ 13482us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:18.942087 (+3044557us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:16.251762 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.252165 (+ 403us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.252190 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:16.252202 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:16.252486 (+ 284us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.252527 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.252544 (+ 17us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:16.252692 (+ 148us) tablet.cc:699] Row locks acquired
0923 17:31:16.252705 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:16.252823 (+ 118us) write_op.cc:270] Start()
0923 17:31:16.252918 (+ 95us) write_op.cc:276] Timestamp: P: 1790184676247393 usec, L: 0
0923 17:31:16.252933 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.253473 (+ 540us) log.cc:842] Serialized 536 byte log entry
0923 17:31:16.287827 (+ 34354us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.288600 (+ 773us) write_op.cc:301] APPLY: starting
0923 17:31:16.288656 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.288761 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.288874 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.289069 (+ 195us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.289214 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.289256 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:16.289442 (+ 186us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.289590 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.289655 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:16.289686 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.289866 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.289894 (+ 28us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:16.332279 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:16.332657 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:16.332678 (+ 21us) write_op.cc:435] Acquired schema lock
0923 17:31:16.332691 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:16.333055 (+ 364us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:16.333106 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:16.333127 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:16.333276 (+ 149us) tablet.cc:699] Row locks acquired
0923 17:31:16.333288 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:16.333394 (+ 106us) write_op.cc:270] Start()
0923 17:31:16.333484 (+ 90us) write_op.cc:276] Timestamp: P: 1790184676327963 usec, L: 0
0923 17:31:16.333500 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:16.522315 (+188815us) log.cc:842] Serialized 356 byte log entry
0923 17:31:16.522896 (+ 581us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:16.602563 (+ 79667us) op_driver.cc:444] REPLICATION: finished
0923 17:31:16.602977 (+ 414us) write_op.cc:301] APPLY: starting
0923 17:31:16.603031 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:16.603119 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:16.603137 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:16.603309 (+ 172us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:16.603434 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:16.603474 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:16.603635 (+ 161us) log.cc:842] Serialized 23 byte log entry
0923 17:31:16.603791 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.603850 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:16.603879 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:16.604053 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:16.604076 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15366,"reported_tablets":16,"spinlock_wait_cycles":24497024,"child_traces":[["op",{"apply.queue_time_us":603,"apply.run_cpu_time_us":1463,"apply.run_wall_time_us":1462,"num_ops":1,"prepare.queue_time_us":18754,"prepare.run_cpu_time_us":2037,"prepare.run_wall_time_us":3232,"raft.queue_time_us":24825,"raft.run_cpu_time_us":833,"raft.run_wall_time_us":834,"replication_time_us":34815}],["op",{"apply.queue_time_us":268,"apply.run_cpu_time_us":1251,"apply.run_wall_time_us":1250,"num_ops":1,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":3577,"prepare.run_wall_time_us":190753,"replication_time_us":268982,"spinlock_wait_cycles":433075456}]]}
W20260923 17:31:19.040562 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a60bf118082b4e9991bfeb3ac59eb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a60bf118082b4e9991bfeb3ac59eb118 already in progress: creating tablet
W20260923 17:31:19.121781 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 81259.5us
I20260923 17:31:19.062580 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:19.129432 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.132961 17144 ts_tablet_manager.cc:1428] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.026s user 0.005s sys 0.001s
I20260923 17:31:19.134349 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:19.137243 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet fba670ca9ed0470e95dc7eb77bca73e4 consensus state has no leader
I20260923 17:31:19.139374 17536 raft_consensus.cc:686] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:19.139057 17144 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.140511 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_93989af63763474587fe6c96b168a123"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:19.142859 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93989af63763474587fe6c96b168a123 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:19.146380 17144 tablet_bootstrap.cc:654] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.140481 17536 consensus_queue.cc:237] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.154548 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_384fc4545b2d47ab8cb690f20bcc27d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:19.164924 17403 ts_tablet_manager.cc:1918] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:19.166939 17403 log.cc:1196] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:19.167763 17403 ts_tablet_manager.cc:1939] T d22f0f9d7e6a46a08ca9f411cc2c658b P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:19.167977 17143 tablet_bootstrap.cc:654] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.173923 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 119) took 1068 ms. Trace:
I20260923 17:31:19.184991 17403 rpcz_store.cc:276] 0923 17:31:18.105961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.106236 (+ 275us) service_pool.cc:227] Handling call
0923 17:31:19.048084 (+941848us) ts_tablet_manager.cc:1145] Deleting tablet d22f0f9d7e6a46a08ca9f411cc2c658b
0923 17:31:19.048129 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:19.102725 (+ 54596us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:19.164692 (+ 61967us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:19.173868 (+ 9176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":940633}
I20260923 17:31:19.186352 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet d22f0f9d7e6a46a08ca9f411cc2c658b (table default.Table_d0e18da1609f485e9684fb7a0228fc35 [id=3027e36914e5452f931466de2316b4cb]) successfully deleted
I20260923 17:31:19.187381 17403 tablet_replica.cc:330] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:19.199555 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 47a0f16066164132a26927f5a830e677 (DEFAULT_TABLE table=default.Table_dc1abdd9ffc44b1d83f6e604bec652df [id=2a56d63c352c458e8d71db88933ee01c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.204854 17403 raft_consensus.cc:2229] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:19.205519 17144 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:19.205596 17403 raft_consensus.cc:2258] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.206010 17144 ts_tablet_manager.cc:1397] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.004s
I20260923 17:31:19.208066 17144 raft_consensus.cc:348] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.208706 17144 raft_consensus.cc:374] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.208942 17144 raft_consensus.cc:729] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.210893 17144 consensus_queue.cc:260] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:19.213433 17565 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.318s user 0.001s sys 0.000s
I20260923 17:31:19.221442 16721 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.225356 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:19.227240 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.229377 16721 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.229877 17144 ts_tablet_manager.cc:1428] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260923 17:31:19.230736 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.239056 17144 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.246142 17403 ts_tablet_manager.cc:1905] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:19.250979 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa62885c564a9bb1ca1b9beacf6e73 (DEFAULT_TABLE table=default.Table_bba5b469220340c781960e02845d372e [id=8212a2a105f74d79b449d9dd03df3bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.252089 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa62885c564a9bb1ca1b9beacf6e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.253003 17144 tablet_bootstrap.cc:654] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.254501 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa62885c564a9bb1ca1b9beacf6e73 (DEFAULT_TABLE table=default.Table_bba5b469220340c781960e02845d372e [id=8212a2a105f74d79b449d9dd03df3bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.255329 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa62885c564a9bb1ca1b9beacf6e73 (DEFAULT_TABLE table=default.Table_bba5b469220340c781960e02845d372e [id=8212a2a105f74d79b449d9dd03df3bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.256403 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa62885c564a9bb1ca1b9beacf6e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.283788 17144 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:19.284248 17144 ts_tablet_manager.cc:1397] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.000s
I20260923 17:31:19.285300 16678 leader_election.cc:302] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.288435 17285 raft_consensus.cc:2781] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:19.288762 17285 raft_consensus.cc:482] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:19.288998 17285 raft_consensus.cc:3037] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:19.300493 16379 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a6894a4ee5fe4884b5162d8055e766fe consensus state has no leader
I20260923 17:31:19.300359 17285 raft_consensus.cc:504] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.303359 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:19.305831 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a0f16066164132a26927f5a830e677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.305585 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:19.306435 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_384fc4545b2d47ab8cb690f20bcc27d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:19.308673 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa62885c564a9bb1ca1b9beacf6e73. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:19.308909 16677 leader_election.cc:341] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.316092 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.318089 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:19.318369 17144 raft_consensus.cc:348] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.319059 17144 raft_consensus.cc:374] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.319303 17144 raft_consensus.cc:729] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.323508 17285 leader_election.cc:288] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.328992 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 1be2b507dfd54f4eb821ea2911b4508f (DEFAULT_TABLE table=default.Table_21e3ee5ab1e640649f6366a299f52319 [id=e118c97abc9e41faaf2384f8abf2ccea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.332983 17144 consensus_queue.cc:260] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.340240 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.341367 16379 master_service.cc:946] ReplaceTablet: received request to replace tablet 547fda6e7bcf474e9f73ba9a6659d34f from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:19.344471 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.361181 17430 consensus_queue.cc:1035] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:19.374498 17144 ts_tablet_manager.cc:1428] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.090s user 0.005s sys 0.000s
W20260923 17:31:19.374590 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 29) took 1337 ms (client timeout 1000 ms). Trace:
W20260923 17:31:19.374976 16382 rpcz_store.cc:269] 0923 17:31:18.037044 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.079144 (+ 42100us) service_pool.cc:227] Handling call
0923 17:31:19.374498 (+1295354us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":147021,"reported_tablets":21}
I20260923 17:31:19.376916 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_a76ca37973d44e81a5631b2e7dc25739" } new_table_name: "default.Table_5626751730cc42bebea406b124700924" modify_external_catalogs: true
I20260923 17:31:19.378813 16721 tablet_service.cc:1467] Processing CreateTablet for tablet 897f08f065fb4bc4bfa03853e4f696bc (DEFAULT_TABLE table=default.Table_cb68aa3dd0ac47d485cd2eb2f7a38165 [id=b689522da9024558bd1829fcba27ad54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.379453 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be2b507dfd54f4eb821ea2911b4508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.381376 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897f08f065fb4bc4bfa03853e4f696bc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:19.354096 16866 consensus_meta.cc:287] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: Time spent flushing consensus metadata: real 0.610s user 0.006s sys 0.000s
I20260923 17:31:19.357996 17143 tablet_bootstrap.cc:492] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:19.383555 17143 ts_tablet_manager.cc:1397] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.508s user 0.023s sys 0.008s
I20260923 17:31:19.385957 17143 raft_consensus.cc:348] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.386607 17143 raft_consensus.cc:374] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.386843 17143 raft_consensus.cc:729] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.388006 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 1be2b507dfd54f4eb821ea2911b4508f (DEFAULT_TABLE table=default.Table_21e3ee5ab1e640649f6366a299f52319 [id=e118c97abc9e41faaf2384f8abf2ccea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.391227 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be2b507dfd54f4eb821ea2911b4508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.392673 17343 ts_tablet_manager.cc:1918] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:19.393045 17343 log.cc:1196] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/051f2e478a324df98647d90107d55533
I20260923 17:31:19.393697 17343 ts_tablet_manager.cc:1939] T 051f2e478a324df98647d90107d55533 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:19.394479 17144 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.394219 17143 consensus_queue.cc:260] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.404064 17542 ts_tablet_manager.cc:1918] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:19.425174 17542 log.cc:1196] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:19.426236 17542 ts_tablet_manager.cc:1939] T 389ecb50e0c34b4d92b5296bb17f4ab1 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:19.429169 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (table default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]) successfully deleted
I20260923 17:31:19.433642 16385 master_service.cc:946] ReplaceTablet: received request to replace tablet a80034cbf6354e1bbe0a0220e43251c3 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:19.415526 17143 ts_tablet_manager.cc:1428] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.032s user 0.001s sys 0.005s
I20260923 17:31:19.447151 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.400537 17146 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:19.412199 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 1be2b507dfd54f4eb821ea2911b4508f (DEFAULT_TABLE table=default.Table_21e3ee5ab1e640649f6366a299f52319 [id=e118c97abc9e41faaf2384f8abf2ccea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.413487 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 897f08f065fb4bc4bfa03853e4f696bc (DEFAULT_TABLE table=default.Table_cb68aa3dd0ac47d485cd2eb2f7a38165 [id=b689522da9024558bd1829fcba27ad54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.450991 17146 ts_tablet_manager.cc:1397] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.650s user 0.014s sys 0.000s
I20260923 17:31:19.451181 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 897f08f065fb4bc4bfa03853e4f696bc (DEFAULT_TABLE table=default.Table_cb68aa3dd0ac47d485cd2eb2f7a38165 [id=b689522da9024558bd1829fcba27ad54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.452538 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897f08f065fb4bc4bfa03853e4f696bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.454347 17146 raft_consensus.cc:348] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.456007 17146 raft_consensus.cc:374] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.456283 17146 raft_consensus.cc:729] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.456950 17146 consensus_queue.cc:260] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.462518 17146 ts_tablet_manager.cc:1428] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260923 17:31:19.462572 17144 tablet_bootstrap.cc:654] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.470487 16725 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:19.480391 16387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 32) took 1339 ms (client timeout 1000 ms). Trace:
W20260923 17:31:19.480942 16387 rpcz_store.cc:269] 0923 17:31:18.141337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.191721 (+ 50384us) service_pool.cc:227] Handling call
0923 17:31:19.480308 (+1288587us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":198963,"reported_tablets":20}
W20260923 17:31:19.482949 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a97e944f4e0c48db813e3bee61102c53 because the tablet was not found. No further retry: Not found: Tablet not found: a97e944f4e0c48db813e3bee61102c53
W20260923 17:31:19.489480 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because the tablet was not found. No further retry: Not found: Tablet not found: 1483835b2bc64873a3570ad17a079713
I20260923 17:31:19.492653 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:19.493317 17040 raft_consensus.cc:3037] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:19.493357 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet d22f0f9d7e6a46a08ca9f411cc2c658b because the tablet was not found. No further retry: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:19.498150 17144 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:19.498565 17144 ts_tablet_manager.cc:1397] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.104s user 0.007s sys 0.001s
I20260923 17:31:19.500790 17144 raft_consensus.cc:348] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.501443 17144 raft_consensus.cc:374] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.501657 17144 raft_consensus.cc:729] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:19.505975 16678 consensus_peers.cc:487] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:19.506721 17285 raft_consensus.cc:482] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.507078 17285 raft_consensus.cc:504] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.508778 17285 leader_election.cc:288] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.510962 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.511616 17041 raft_consensus.cc:2454] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:19.513345 16678 leader_election.cc:302] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:19.514091 17473 raft_consensus.cc:2781] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:19.514405 17473 raft_consensus.cc:482] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:19.514683 17473 raft_consensus.cc:3037] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.518826 17040 raft_consensus.cc:2454] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:19.521625 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be2b507dfd54f4eb821ea2911b4508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.526248 17343 tablet_replica.cc:330] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:19.527088 17343 raft_consensus.cc:2229] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:19.527508 17343 raft_consensus.cc:2258] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.528795 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897f08f065fb4bc4bfa03853e4f696bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.528883 17146 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:19.535938 17146 tablet_bootstrap.cc:654] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.536927 16678 leader_election.cc:302] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.539711 17528 consensus_queue.cc:1035] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:19.539995 17285 raft_consensus.cc:2781] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:19.544914 17285 raft_consensus.cc:686] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:19.545686 17285 consensus_queue.cc:237] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.550333 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 051f2e478a324df98647d90107d55533 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
W20260923 17:31:19.554973 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because the tablet was not found. No further retry: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:19.542282 17343 ts_tablet_manager.cc:1905] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:19.560906 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet d22f0f9d7e6a46a08ca9f411cc2c658b because the tablet was not found. No further retry: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b
W20260923 17:31:19.562475 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a60bf118082b4e9991bfeb3ac59eb118 because the tablet was not found. No further retry: Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:19.546615 17473 raft_consensus.cc:504] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:19.568431 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 723000dc175844e990740dc503b53ced because the tablet was not found. No further retry: Not found: Tablet not found: 723000dc175844e990740dc503b53ced
W20260923 17:31:19.570441 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a3bcaddfad4f45739a705c17bdfe332f because the tablet was not found. No further retry: Not found: Tablet not found: a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:19.463380 17143 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:19.470728 17514 consensus_queue.cc:1035] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:19.588909 17473 leader_election.cc:288] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.503911 17144 consensus_queue.cc:260] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.494225 17542 tablet_replica.cc:330] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:19.509753 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:19.592440 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:19.593073 17040 raft_consensus.cc:3037] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:19.593029 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a97e944f4e0c48db813e3bee61102c53 because the tablet was not found. No further retry: Not found: Tablet not found: a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:19.584079 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_516ec99807654f5385fc92422c51562a" } modify_external_catalogs: true
I20260923 17:31:19.594095 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_516ec99807654f5385fc92422c51562a" } modify_external_catalogs: true
I20260923 17:31:19.600430 17542 raft_consensus.cc:2229] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:19.600957 17542 raft_consensus.cc:2258] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.602967 17040 raft_consensus.cc:2454] T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:19.603775 17542 ts_tablet_manager.cc:1905] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:19.568925 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:19.606865 17430 raft_consensus.cc:482] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.607350 17430 raft_consensus.cc:504] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.609228 17430 leader_election.cc:288] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:19.611845 16678 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:19.613533 16873 tablet_service.cc:1467] Processing CreateTablet for tablet c7e6178ab07245e3a687d53eb2ff03ab (DEFAULT_TABLE table=default.Table_ce218dced6994036a697bb38f509808b [id=52725612173d46beae61a76b2de74331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.614698 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e6178ab07245e3a687d53eb2ff03ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.615262 17144 ts_tablet_manager.cc:1428] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.116s user 0.007s sys 0.000s
W20260923 17:31:19.617749 16677 leader_election.cc:341] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:19.618615 16677 leader_election.cc:341] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.620415 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c842eff86634bf4bc70c426b5226d66" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:19.625536 16677 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:19.625912 16717 consensus_meta.cc:287] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: Time spent flushing consensus metadata: real 0.719s user 0.007s sys 0.001s
I20260923 17:31:19.629288 17536 raft_consensus.cc:482] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.629781 17536 raft_consensus.cc:504] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.633261 17536 leader_election.cc:288] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:19.633889 17146 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:19.634320 17146 ts_tablet_manager.cc:1397] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.001s
I20260923 17:31:19.636538 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547fda6e7bcf474e9f73ba9a6659d34f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:19.638090 16972 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.648569 17143 tablet_bootstrap.cc:654] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.660064 17146 raft_consensus.cc:348] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.660821 17146 raft_consensus.cc:374] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.661072 17146 raft_consensus.cc:729] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.661746 17146 consensus_queue.cc:260] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.626231 16677 leader_election.cc:302] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:19.635035 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "547fda6e7bcf474e9f73ba9a6659d34f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:19.639385 17144 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.651750 16726 tablet_service.cc:1467] Processing CreateTablet for tablet c7e6178ab07245e3a687d53eb2ff03ab (DEFAULT_TABLE table=default.Table_ce218dced6994036a697bb38f509808b [id=52725612173d46beae61a76b2de74331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.666146 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e6178ab07245e3a687d53eb2ff03ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:19.666330 16972 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.666741 16972 leader_election.cc:302] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.666921 17285 raft_consensus.cc:2726] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:19.653113 16678 leader_election.cc:302] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.668987 17285 raft_consensus.cc:2781] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:19.669668 17285 raft_consensus.cc:686] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:19.670667 17285 consensus_queue.cc:237] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.679811 17144 tablet_bootstrap.cc:654] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.686463 16388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 170) took 1149 ms. Trace:
I20260923 17:31:19.689288 16379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:19.696817 16379 catalog_manager.cc:6565] ReplaceTablet: tablet 547fda6e7bcf474e9f73ba9a6659d34f deleted and replaced by tablet 006895667f324d049debbebb132ed09b
I20260923 17:31:19.686961 16388 rpcz_store.cc:276] 0923 17:31:18.537024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.023075 (+486051us) service_pool.cc:227] Handling call
0923 17:31:19.100812 (+ 77737us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:19.102471 (+ 1659us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:19.403360 (+300889us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b5400011680. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f91279a4361 00007f9127a74a86 00007f9127a6ae03 00007f9127a2fce7 00007f9127a2faf4 00007f91296101f1 00007f9129610a16 00007f91294bd785 00007f91295df18f 00007f912711c034 00007f912711bfc2
0923 17:31:19.685762 (+282402us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:19.686099 (+ 337us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:19.686372 (+ 273us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:19.403655 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:19.404007 (+ 352us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:19.404029 (+ 22us) write_op.cc:435] Acquired schema lock
0923 17:31:19.404042 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:19.444847 (+ 40805us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:19.444890 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:19.444911 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:19.445114 (+ 203us) tablet.cc:699] Row locks acquired
0923 17:31:19.445129 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:19.445260 (+ 131us) write_op.cc:270] Start()
0923 17:31:19.445378 (+ 118us) write_op.cc:276] Timestamp: P: 1790184679445189 usec, L: 0
0923 17:31:19.445395 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:19.586141 (+140746us) log.cc:842] Serialized 433 byte log entry
0923 17:31:19.595086 (+ 8945us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:19.679580 (+ 84494us) op_driver.cc:444] REPLICATION: finished
0923 17:31:19.680418 (+ 838us) write_op.cc:301] APPLY: starting
0923 17:31:19.680486 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:19.680620 (+ 134us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:19.680638 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:19.681042 (+ 404us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:19.681195 (+ 153us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:19.681249 (+ 54us) write_op.cc:312] APPLY: finished
0923 17:31:19.681457 (+ 208us) log.cc:842] Serialized 29 byte log entry
0923 17:31:19.681622 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:19.681697 (+ 75us) write_op.cc:454] Released schema lock
0923 17:31:19.681728 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:19.681917 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:19.681939 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":176555392,"child_traces":[["op",{"apply.queue_time_us":655,"apply.run_cpu_time_us":1701,"apply.run_wall_time_us":1701,"num_ops":2,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":3631,"prepare.run_wall_time_us":191619,"raft.queue_time_us":26115,"raft.run_cpu_time_us":970,"raft.run_wall_time_us":35021,"replication_time_us":234078,"spinlock_wait_cycles":322052992}]]}
I20260923 17:31:19.701159 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (replaced by tablet 006895667f324d049debbebb132ed09b) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:19.717500 17586 log.cc:924] Time spent T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.087s user 0.001s sys 0.000s
W20260923 17:31:19.719254 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 52962.1us
I20260923 17:31:19.720695 17536 raft_consensus.cc:2726] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:19.711459 17021 tablet_service.cc:1467] Processing CreateTablet for tablet c7e6178ab07245e3a687d53eb2ff03ab (DEFAULT_TABLE table=default.Table_ce218dced6994036a697bb38f509808b [id=52725612173d46beae61a76b2de74331]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:19.721961 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e6178ab07245e3a687d53eb2ff03ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:19.724377 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (replaced by tablet 006895667f324d049debbebb132ed09b) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.724660 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (replaced by tablet 006895667f324d049debbebb132ed09b) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:19.726995 17565 tablet.cc:1722] T cbb9840f8b6d464caae440fb03a7be54 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:19.733196 17146 ts_tablet_manager.cc:1428] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.099s user 0.007s sys 0.000s
I20260923 17:31:19.735911 17146 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:19.743748 17343 ts_tablet_manager.cc:1918] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:19.744163 17343 log.cc:1196] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:19.744755 17343 ts_tablet_manager.cc:1939] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:19.759982 17143 tablet_bootstrap.cc:492] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:19.760426 17143 ts_tablet_manager.cc:1397] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.297s user 0.013s sys 0.014s
I20260923 17:31:19.762586 17143 raft_consensus.cc:348] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.763252 17143 raft_consensus.cc:374] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.763500 17143 raft_consensus.cc:729] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.772526 17430 raft_consensus.cc:482] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.773018 17430 raft_consensus.cc:504] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.774981 17430 leader_election.cc:288] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.776454 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:19.780917 17143 consensus_queue.cc:260] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.783792 17571 raft_consensus.cc:482] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.784243 17571 raft_consensus.cc:504] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:19.785131 16677 leader_election.cc:341] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.787055 17600 raft_consensus.cc:482] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.787467 17600 raft_consensus.cc:504] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.788502 17571 leader_election.cc:288] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.793740 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.794464 17040 raft_consensus.cc:2379] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:19.793751 17597 tablet.cc:1722] T cbb9840f8b6d464caae440fb03a7be54 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:19.796952 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.797607 17039 raft_consensus.cc:2454] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:19.801910 17143 ts_tablet_manager.cc:1428] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.041s user 0.004s sys 0.001s
I20260923 17:31:19.803375 17343 tablet_replica.cc:330] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:19.804076 17343 raft_consensus.cc:2229] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:19.804961 17343 pending_rounds.cc:62] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:19.798094 17144 tablet_bootstrap.cc:492] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:19.801010 16678 leader_election.cc:302] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.801913 17473 consensus_queue.cc:1035] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:19.803747 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c204e5fffe4f490295a12b8c93818c3c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:19.807772 16826 leader_election.cc:302] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:19.808171 17430 raft_consensus.cc:2781] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:19.808549 17430 raft_consensus.cc:482] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:19.808732 17571 raft_consensus.cc:3037] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.808838 17430 raft_consensus.cc:3037] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.816098 17571 raft_consensus.cc:2726] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:19.816762 17144 ts_tablet_manager.cc:1397] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.178s user 0.011s sys 0.001s
I20260923 17:31:19.817636 17143 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:19.818025 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:19.819564 17600 leader_election.cc:288] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:19.823153 16677 leader_election.cc:341] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.826665 17430 raft_consensus.cc:504] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.832046 17146 tablet_bootstrap.cc:654] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.838238 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.838176 17601 tablet.cc:1722] T cbb9840f8b6d464caae440fb03a7be54 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:19.839715 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (table default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]) successfully deleted
I20260923 17:31:19.840567 17040 raft_consensus.cc:2454] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:19.841347 16386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_2d35179a997541eda78f604ede819340" } new_table_name: "default.Table_632f010f9d114ecbab7901aa664556ac" modify_external_catalogs: true
I20260923 17:31:19.845801 16379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_76bda21bbdc644518bea95216cd34ea7" } modify_external_catalogs: true
I20260923 17:31:19.846474 16379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_76bda21bbdc644518bea95216cd34ea7" } modify_external_catalogs: true
I20260923 17:31:19.851007 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_e890a913f28f4a3b935f9b3ebfd75b9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:19.852906 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e890a913f28f4a3b935f9b3ebfd75b9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:19.859552 17143 tablet_bootstrap.cc:654] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.865193 17542 ts_tablet_manager.cc:1918] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:19.865806 17542 log.cc:1196] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:19.866593 17542 ts_tablet_manager.cc:1939] T 8d65cfb2eb3648faab5869bba69cab3f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:19.869441 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 8d65cfb2eb3648faab5869bba69cab3f (table default.Table_28db4e4c622f47c9838b6d44c68d5499 [id=8ffd6a94b3b142e988bef11b75aa1766]) successfully deleted
I20260923 17:31:19.872509 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.842542 16678 leader_election.cc:302] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:19.805253 17343 pending_rounds.cc:68] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596445814140928 op_type: NO_OP noop_request { }
I20260923 17:31:19.879104 17343 raft_consensus.cc:2866] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:19.879045 17528 raft_consensus.cc:2781] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:19.879403 17528 raft_consensus.cc:482] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:19.879617 17343 raft_consensus.cc:2258] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.879753 17528 raft_consensus.cc:3037] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.881404 17430 leader_election.cc:288] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.849563 17144 raft_consensus.cc:348] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.882102 17144 raft_consensus.cc:374] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.882061 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:19.882436 17144 raft_consensus.cc:729] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.882777 17041 raft_consensus.cc:3037] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.883067 17144 consensus_queue.cc:260] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.882954 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:19.885293 17144 ts_tablet_manager.cc:1428] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.067s user 0.005s sys 0.000s
W20260923 17:31:19.885339 16677 leader_election.cc:341] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.854139 17565 tablet.cc:1722] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260923 17:31:19.901846 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 29d44ee57faf4a1d99e077fd781a519c because the tablet was not found. No further retry: Not found: Tablet not found: 29d44ee57faf4a1d99e077fd781a519c
I20260923 17:31:19.902997 17542 tablet_replica.cc:330] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:19.905555 17430 raft_consensus.cc:482] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.906107 17430 raft_consensus.cc:504] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.899349 17143 tablet_bootstrap.cc:492] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:19.910532 17143 ts_tablet_manager.cc:1397] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.093s user 0.011s sys 0.001s
I20260923 17:31:19.910341 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:19.910887 17430 leader_election.cc:288] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.911314 16893 raft_consensus.cc:2454] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:19.912801 16677 leader_election.cc:302] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:19.912766 17143 raft_consensus.cc:348] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.913450 17143 raft_consensus.cc:374] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:19.913672 17143 raft_consensus.cc:729] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:19.914054 17430 raft_consensus.cc:2781] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:19.913831 17542 raft_consensus.cc:2229] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:19.915787 17542 raft_consensus.cc:2258] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:19.914829 17430 raft_consensus.cc:482] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:19.916450 16743 raft_consensus.cc:3037] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.917872 17528 raft_consensus.cc:504] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.920535 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:19.926187 17144 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:19.928615 17343 ts_tablet_manager.cc:1905] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:19.914304 17143 consensus_queue.cc:260] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.927903 17536 raft_consensus.cc:482] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.939000 17536 raft_consensus.cc:504] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.943185 17536 leader_election.cc:288] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:19.945950 17528 leader_election.cc:288] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:19.947858 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.948463 17013 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.949091 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.950278 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba670ca9ed0470e95dc7eb77bca73e4" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:19.950789 17040 raft_consensus.cc:3037] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.953233 17144 tablet_bootstrap.cc:654] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:19.916325 17430 raft_consensus.cc:3037] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.956703 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:19.924898 17571 consensus_queue.cc:1035] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260923 17:31:19.957299 17036 raft_consensus.cc:2454] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:19.957683 17403 ts_tablet_manager.cc:1918] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:19.949131 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:19.960592 16972 leader_election.cc:341] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:19.964464 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:19.965082 17536 raft_consensus.cc:482] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:19.965576 17536 raft_consensus.cc:504] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.943643 16893 raft_consensus.cc:3037] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:19.941932 17143 ts_tablet_manager.cc:1428] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260923 17:31:19.976511 17143 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:19.952437 17606 tablet.cc:1722] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:19.924185 17542 ts_tablet_manager.cc:1905] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:19.891907 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 25) took 1615 ms (client timeout 1000 ms). Trace:
W20260923 17:31:19.993995 16384 rpcz_store.cc:269] 0923 17:31:18.276319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.300170 (+ 23851us) service_pool.cc:227] Handling call
0923 17:31:19.891830 (+1591660us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":29}
I20260923 17:31:20.002869 16893 raft_consensus.cc:2454] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:20.019829 17536 leader_election.cc:288] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:19.993144 17040 raft_consensus.cc:2454] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:20.022207 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:20.023797 16972 leader_election.cc:341] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.024255 17146 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:20.024705 17146 ts_tablet_manager.cc:1397] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.291s user 0.010s sys 0.005s
I20260923 17:31:20.026818 17146 raft_consensus.cc:348] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:19.976195 17041 raft_consensus.cc:2454] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
W20260923 17:31:19.997943 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e329dd46a3a245b6826b01ac8de56621 consensus state has no leader
I20260923 17:31:20.047313 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_2b5ca9111a5a4876bf1ac5b3c3f857a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:20.049190 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b5ca9111a5a4876bf1ac5b3c3f857a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.050060 17146 raft_consensus.cc:374] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.050417 17146 raft_consensus.cc:729] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.051259 17146 consensus_queue.cc:260] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.000370 16385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:20.055546 16385 catalog_manager.cc:6565] ReplaceTablet: tablet a80034cbf6354e1bbe0a0220e43251c3 deleted and replaced by tablet 7c115e29673747f8b0712d441ac993fb
I20260923 17:31:20.056238 17146 ts_tablet_manager.cc:1428] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.031s user 0.004s sys 0.001s
I20260923 17:31:20.056983 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:20.057446 17146 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:20.056833 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_1fff36e7ada04065a0972c828cd7956b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:20.058633 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fff36e7ada04065a0972c828cd7956b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.002041 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:20.062917 17143 tablet_bootstrap.cc:654] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.062889 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:20.031193 16379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 261d1d03290546969a74ca60beb890eb
I20260923 17:31:20.072129 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:20.083453 17343 ts_tablet_manager.cc:1918] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.083076 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_ca364ecd26d844f8b7394a4f3770b1b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:20.084059 17343 log.cc:1196] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:20.084762 17343 ts_tablet_manager.cc:1939] T a6f62eadc6a94aedb4a8164c545cda17 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
W20260923 17:31:20.084736 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca364ecd26d844f8b7394a4f3770b1b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.084980 17534 consensus_queue.cc:1035] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:19.969910 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.107929 17403 log.cc:1196] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:20.007264 17430 raft_consensus.cc:504] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.108913 17403 ts_tablet_manager.cc:1939] T a97e944f4e0c48db813e3bee61102c53 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:20.054088 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:20.066013 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:20.070719 17144 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:19.983506 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.115609 17144 ts_tablet_manager.cc:1397] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.190s user 0.008s sys 0.015s
I20260923 17:31:20.109683 17430 leader_election.cc:288] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.117784 17144 raft_consensus.cc:348] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.095027 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:20.118400 17144 raft_consensus.cc:374] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.096486 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:20.101716 16380 catalog_manager.cc:5495] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:20.118613 17144 raft_consensus.cc:729] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:20.107352 16972 leader_election.cc:341] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.022049 16678 leader_election.cc:302] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:20.118927 16972 leader_election.cc:302] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.119015 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
W20260923 17:31:20.119345 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 97383.7us
I20260923 17:31:20.119619 17536 raft_consensus.cc:2726] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:20.119918 16972 leader_election.cc:341] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.120536 16972 leader_election.cc:302] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.121447 17536 raft_consensus.cc:2726] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.121428 16678 leader_election.cc:302] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.122503 17285 raft_consensus.cc:2781] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:20.122876 17285 raft_consensus.cc:686] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:20.123587 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:20.123492 17285 consensus_queue.cc:237] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.124189 17040 raft_consensus.cc:3037] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.128803 17144 consensus_queue.cc:260] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.131783 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105455d69eb64adc8513b1bfa97e6330" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:20.128041 17603 raft_consensus.cc:2781] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:20.133092 17603 raft_consensus.cc:686] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:20.134267 16885 raft_consensus.cc:3037] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.134234 17597 tablet.cc:1722] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:20.133792 17603 consensus_queue.cc:237] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.138979 17144 ts_tablet_manager.cc:1428] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.023s user 0.000s sys 0.005s
I20260923 17:31:20.139750 17144 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:20.142902 17040 raft_consensus.cc:2454] T 105455d69eb64adc8513b1bfa97e6330 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:20.144866 16678 leader_election.cc:302] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:20.145843 17528 raft_consensus.cc:2781] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:20.146239 17528 raft_consensus.cc:686] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:20.146926 17528 consensus_queue.cc:237] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.149777 16885 raft_consensus.cc:2454] T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:20.152869 17146 tablet_bootstrap.cc:654] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:20.154273 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 56634.1us
W20260923 17:31:20.155216 16373 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 85eba2aac8d84defa8ce4081d447bc63 consensus state has no leader
I20260923 17:31:20.155531 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 118) took 2006 ms. Trace:
I20260923 17:31:20.167995 17143 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:20.168418 17343 tablet_replica.cc:330] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:20.168450 17143 ts_tablet_manager.cc:1397] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.192s user 0.006s sys 0.006s
I20260923 17:31:20.156489 17403 rpcz_store.cc:276] 0923 17:31:18.105156 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.246608 (+141452us) service_pool.cc:227] Handling call
0923 17:31:19.187225 (+940617us) ts_tablet_manager.cc:1145] Deleting tablet a97e944f4e0c48db813e3bee61102c53
0923 17:31:19.187271 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:19.620188 (+432917us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:19.957447 (+337259us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.111472 (+154025us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16587,"tablet-delete.queue_time_us":939913}
I20260923 17:31:20.169457 17343 raft_consensus.cc:2229] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.169657 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a97e944f4e0c48db813e3bee61102c53 (table default.Table_4c7c906814484e6e95b4066c4ce97fb7 [id=2113acd5821a4de1bf1f5a74fd7e44b4]) successfully deleted
I20260923 17:31:20.169932 17343 raft_consensus.cc:2258] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.170770 17143 raft_consensus.cc:348] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.171429 17143 raft_consensus.cc:374] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.171444 17343 ts_tablet_manager.cc:1905] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.200421 17146 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:20.200825 17146 ts_tablet_manager.cc:1397] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.144s user 0.004s sys 0.005s
I20260923 17:31:20.204967 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a6f62eadc6a94aedb4a8164c545cda17 (table default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]) successfully deleted
I20260923 17:31:20.214046 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 121) took 1526 ms. Trace:
I20260923 17:31:20.215142 17146 raft_consensus.cc:348] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.215797 17146 raft_consensus.cc:374] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.216063 17146 raft_consensus.cc:729] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.171658 17143 raft_consensus.cc:729] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.218829 17143 consensus_queue.cc:260] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.220919 17144 tablet_bootstrap.cc:654] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.220995 17143 ts_tablet_manager.cc:1428] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260923 17:31:20.216938 17403 rpcz_store.cc:276] 0923 17:31:18.688278 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.688448 (+ 170us) service_pool.cc:227] Handling call
0923 17:31:20.205539 (+1517091us) ts_tablet_manager.cc:1145] Deleting tablet 051f2e478a324df98647d90107d55533
0923 17:31:20.208976 (+ 3437us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.213982 (+ 5006us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1516455}
W20260923 17:31:20.224038 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 051f2e478a324df98647d90107d55533 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 051f2e478a324df98647d90107d55533 already in progress: creating tablet
I20260923 17:31:20.223273 17143 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:20.226419 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 127) took 1267 ms. Trace:
W20260923 17:31:20.231611 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260923 17:31:20.232523 17146 consensus_queue.cc:260] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.246254 17146 ts_tablet_manager.cc:1428] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260923 17:31:20.227924 17403 rpcz_store.cc:276] 0923 17:31:18.959530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.960858 (+ 1328us) service_pool.cc:227] Handling call
0923 17:31:20.226118 (+1265260us) ts_tablet_manager.cc:1145] Deleting tablet 1483835b2bc64873a3570ad17a079713
0923 17:31:20.226174 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.226359 (+ 185us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1263447}
I20260923 17:31:20.259032 16385 master_service.cc:946] ReplaceTablet: received request to replace tablet 42ccd82f191942239ffb88459d362107 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:20.266534 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 128) took 1306 ms. Trace:
I20260923 17:31:20.267858 17146 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:20.270268 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1483835b2bc64873a3570ad17a079713 because the tablet was not found. No further retry: Not found: Tablet not found: 1483835b2bc64873a3570ad17a079713
I20260923 17:31:20.258914 17144 tablet_bootstrap.cc:492] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.277931 17144 ts_tablet_manager.cc:1397] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.138s user 0.008s sys 0.004s
I20260923 17:31:20.285017 17143 tablet_bootstrap.cc:654] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.290395 17343 ts_tablet_manager.cc:1918] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:20.290761 17343 log.cc:1196] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:20.291316 17343 ts_tablet_manager.cc:1939] T 547fda6e7bcf474e9f73ba9a6659d34f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:20.294970 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:20.266885 17403 rpcz_store.cc:276] 0923 17:31:18.959838 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.961521 (+ 1683us) service_pool.cc:227] Handling call
0923 17:31:20.266230 (+1304709us) ts_tablet_manager.cc:1145] Deleting tablet 0721fa178c0347598351b9d3512089b8
0923 17:31:20.266278 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.266470 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1304253}
W20260923 17:31:20.296298 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 0721fa178c0347598351b9d3512089b8 because the tablet was not found. No further retry: Not found: Tablet not found: 0721fa178c0347598351b9d3512089b8
I20260923 17:31:20.296799 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 129) took 1336 ms. Trace:
I20260923 17:31:20.297792 17403 rpcz_store.cc:276] 0923 17:31:18.960116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.962038 (+ 1922us) service_pool.cc:227] Handling call
0923 17:31:20.296402 (+1334364us) ts_tablet_manager.cc:1145] Deleting tablet 8879dee684214c6dac3f9d6642a6d97d
0923 17:31:20.296448 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.296740 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1333904}
W20260923 17:31:20.299249 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 8879dee684214c6dac3f9d6642a6d97d because the tablet was not found. No further retry: Not found: Tablet not found: 8879dee684214c6dac3f9d6642a6d97d
I20260923 17:31:20.300523 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 130) took 1340 ms. Trace:
I20260923 17:31:20.300813 17403 rpcz_store.cc:276] 0923 17:31:18.960427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.962589 (+ 2162us) service_pool.cc:227] Handling call
0923 17:31:20.300270 (+1337681us) ts_tablet_manager.cc:1145] Deleting tablet 21a629ce6ab14d12b6a83ce6bc56f70f
0923 17:31:20.300316 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.300470 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1337207}
W20260923 17:31:20.301759 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 21a629ce6ab14d12b6a83ce6bc56f70f because the tablet was not found. No further retry: Not found: Tablet not found: 21a629ce6ab14d12b6a83ce6bc56f70f
I20260923 17:31:20.289719 17542 ts_tablet_manager.cc:1918] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.303004 17542 log.cc:1196] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/723000dc175844e990740dc503b53ced
I20260923 17:31:20.303287 16382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_1fff36e7ada04065a0972c828cd7956b" } modify_external_catalogs: true
I20260923 17:31:20.303779 16382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_1fff36e7ada04065a0972c828cd7956b" } modify_external_catalogs: true
I20260923 17:31:20.304462 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 131) took 1343 ms. Trace:
I20260923 17:31:20.303812 17542 ts_tablet_manager.cc:1939] T 723000dc175844e990740dc503b53ced P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:20.304736 17403 rpcz_store.cc:276] 0923 17:31:18.960702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.963106 (+ 2404us) service_pool.cc:227] Handling call
0923 17:31:20.304183 (+1341077us) ts_tablet_manager.cc:1145] Deleting tablet a3bcaddfad4f45739a705c17bdfe332f
0923 17:31:20.304253 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.304410 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1340652}
W20260923 17:31:20.307636 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a3bcaddfad4f45739a705c17bdfe332f because the tablet was not found. No further retry: Not found: Tablet not found: a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:20.302480 16384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_fcfadb32897349d4962baa0dc45bd2da" } new_table_name: "default.Table_dcf1d5122d0c421bab95f4f6e66d3f47" modify_external_catalogs: true
I20260923 17:31:20.309201 17403 tablet_replica.cc:330] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:20.312995 17146 tablet_bootstrap.cc:654] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.318594 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:20.319984 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:20.280206 17144 raft_consensus.cc:348] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.321470 17144 raft_consensus.cc:374] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.321764 17144 raft_consensus.cc:729] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.328279 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 132) took 1340 ms. Trace:
I20260923 17:31:20.328764 17542 rpcz_store.cc:276] 0923 17:31:18.987562 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.002402 (+ 14840us) service_pool.cc:227] Handling call
0923 17:31:19.902824 (+900422us) ts_tablet_manager.cc:1145] Deleting tablet 723000dc175844e990740dc503b53ced
0923 17:31:19.902865 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.172101 (+269236us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.289435 (+117334us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.328172 (+ 38737us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":899805}
I20260923 17:31:20.330288 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 723000dc175844e990740dc503b53ced (table default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]) successfully deleted
I20260923 17:31:20.330659 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 133) took 1327 ms. Trace:
I20260923 17:31:20.330952 17542 rpcz_store.cc:276] 0923 17:31:19.003643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.003768 (+ 125us) service_pool.cc:227] Handling call
0923 17:31:20.330359 (+1326591us) ts_tablet_manager.cc:1145] Deleting tablet a3bcaddfad4f45739a705c17bdfe332f
0923 17:31:20.330407 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.330591 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1326093}
I20260923 17:31:20.331363 17144 consensus_queue.cc:260] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:20.331993 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a3bcaddfad4f45739a705c17bdfe332f because the tablet was not found. No further retry: Not found: Tablet not found: a3bcaddfad4f45739a705c17bdfe332f
I20260923 17:31:20.332070 17403 raft_consensus.cc:2229] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:20.333069 17403 raft_consensus.cc:2258] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.335908 17403 ts_tablet_manager.cc:1905] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.353478 17144 ts_tablet_manager.cc:1428] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.075s user 0.005s sys 0.000s
I20260923 17:31:20.354367 17144 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:20.356169 17146 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:20.356567 17146 ts_tablet_manager.cc:1397] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.089s user 0.014s sys 0.000s
I20260923 17:31:20.359221 17146 raft_consensus.cc:348] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.359812 17146 raft_consensus.cc:374] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.368834 16382 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 3e9d46727582416a87fb828bd1fb4ba5
I20260923 17:31:20.372385 16383 catalog_manager.cc:5495] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:20.374027 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:20.374343 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 547fda6e7bcf474e9f73ba9a6659d34f (table default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]) successfully deleted
W20260923 17:31:20.375065 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because the tablet was not found. No further retry: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:20.375576 17542 tablet_replica.cc:330] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:20.375813 17343 tablet_replica.cc:330] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:20.376487 17542 raft_consensus.cc:2229] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.376672 17343 raft_consensus.cc:2229] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.377020 17542 raft_consensus.cc:2258] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.377290 17343 raft_consensus.cc:2258] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.379007 17542 ts_tablet_manager.cc:1905] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.389776 17143 tablet_bootstrap.cc:492] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:20.390390 17143 ts_tablet_manager.cc:1397] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.167s user 0.009s sys 0.002s
I20260923 17:31:20.392954 17143 raft_consensus.cc:348] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.398520 17143 raft_consensus.cc:374] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.398821 17143 raft_consensus.cc:729] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.399487 17143 consensus_queue.cc:260] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.360051 17146 raft_consensus.cc:729] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.405159 17146 consensus_queue.cc:260] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.407146 17146 ts_tablet_manager.cc:1428] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260923 17:31:20.416813 17536 raft_consensus.cc:482] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.417222 17536 raft_consensus.cc:504] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.418947 17536 leader_election.cc:288] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:20.420899 17144 tablet_bootstrap.cc:654] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.421147 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:20.422962 16972 leader_election.cc:341] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.421100 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:20.428375 16972 leader_election.cc:341] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.428813 17143 ts_tablet_manager.cc:1428] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20260923 17:31:20.428777 16972 leader_election.cc:302] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.430824 17536 raft_consensus.cc:2726] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.438055 17143 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:20.440099 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_0cad32e2a8eb4f078e9be43733d0b50d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:20.441733 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cad32e2a8eb4f078e9be43733d0b50d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.379112 17343 ts_tablet_manager.cc:1905] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:20.453944 16677 consensus_peers.cc:487] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:20.461849 17146 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:20.466463 17143 tablet_bootstrap.cc:654] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.471848 17146 tablet_bootstrap.cc:654] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.479187 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_4de36798f891433eb8cf23065562cfc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:20.481528 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4de36798f891433eb8cf23065562cfc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.491063 17144 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.491484 17144 ts_tablet_manager.cc:1397] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.137s user 0.014s sys 0.002s
I20260923 17:31:20.493531 17144 raft_consensus.cc:348] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.494165 17144 raft_consensus.cc:374] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.494398 17144 raft_consensus.cc:729] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.495114 17144 consensus_queue.cc:260] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.511432 17144 ts_tablet_manager.cc:1428] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260923 17:31:20.500036 17542 ts_tablet_manager.cc:1918] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.514988 17542 log.cc:1196] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:20.515811 17542 ts_tablet_manager.cc:1939] T 3b9b734b250d4920abbafe76f52ec540 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:20.516285 17536 raft_consensus.cc:482] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.517904 17536 raft_consensus.cc:504] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.519731 17536 leader_election.cc:288] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:20.521417 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.522037 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:20.522428 17571 raft_consensus.cc:482] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.522892 17571 raft_consensus.cc:504] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:20.523505 16972 leader_election.cc:341] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.523828 17144 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:20.526294 16972 leader_election.cc:341] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.526684 16972 leader_election.cc:302] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.526950 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 134) took 1522 ms. Trace:
I20260923 17:31:20.527278 17542 rpcz_store.cc:276] 0923 17:31:19.004690 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.004793 (+ 103us) service_pool.cc:227] Handling call
0923 17:31:20.375383 (+1370590us) ts_tablet_manager.cc:1145] Deleting tablet 3b9b734b250d4920abbafe76f52ec540
0923 17:31:20.375429 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.470015 (+ 94586us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.499758 (+ 29743us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.526876 (+ 27118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1370099}
I20260923 17:31:20.528430 17571 leader_election.cc:288] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.528857 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 3b9b734b250d4920abbafe76f52ec540 (table default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]) successfully deleted
I20260923 17:31:20.529425 17144 tablet_bootstrap.cc:654] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.529836 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:20.530679 17040 raft_consensus.cc:2379] T 6dbfc01e66404a5da4f30c5eff8e7f45 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:20.532364 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbfc01e66404a5da4f30c5eff8e7f45" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.533594 17536 raft_consensus.cc:2726] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.533828 16826 leader_election.cc:302] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:20.534933 17571 raft_consensus.cc:3037] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.541983 17571 raft_consensus.cc:2726] T 6dbfc01e66404a5da4f30c5eff8e7f45 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.545434 17542 tablet_replica.cc:330] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:20.547216 17542 raft_consensus.cc:2229] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.549719 17144 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.550135 17144 ts_tablet_manager.cc:1397] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.001s
I20260923 17:31:20.550742 17542 raft_consensus.cc:2258] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.552738 17144 raft_consensus.cc:348] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.553646 17144 raft_consensus.cc:374] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.553901 17144 raft_consensus.cc:729] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.557848 17143 tablet_bootstrap.cc:492] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:20.558120 17146 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:20.558444 17143 ts_tablet_manager.cc:1397] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.121s user 0.013s sys 0.009s
I20260923 17:31:20.558547 17146 ts_tablet_manager.cc:1397] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.000s
I20260923 17:31:20.560577 17144 consensus_queue.cc:260] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.570045 17542 ts_tablet_manager.cc:1905] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.570436 17146 raft_consensus.cc:348] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.571130 17146 raft_consensus.cc:374] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.571379 17146 raft_consensus.cc:729] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.575374 17144 ts_tablet_manager.cc:1428] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260923 17:31:20.561148 17143 raft_consensus.cc:348] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.578812 17143 raft_consensus.cc:374] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.579205 17143 raft_consensus.cc:729] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.567488 16382 master_service.cc:946] ReplaceTablet: received request to replace tablet e818f6192de6472cb27747c0e3a1f128 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:20.580003 17143 consensus_queue.cc:260] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.582639 17343 ts_tablet_manager.cc:1918] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:20.583063 17343 log.cc:1196] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:20.584765 17343 ts_tablet_manager.cc:1939] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:20.588025 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (table default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]) successfully deleted
I20260923 17:31:20.572054 17146 consensus_queue.cc:260] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.592948 17536 raft_consensus.cc:482] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.593343 17536 raft_consensus.cc:504] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.594123 17144 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:20.598259 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.599221 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:20.600092 16972 leader_election.cc:341] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:20.600808 16972 leader_election.cc:341] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.601169 16972 leader_election.cc:302] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.600090 17536 leader_election.cc:288] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:20.606917 17144 tablet_bootstrap.cc:654] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.607982 17143 ts_tablet_manager.cc:1428] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260923 17:31:20.609745 17536 raft_consensus.cc:2726] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:20.615741 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a97e944f4e0c48db813e3bee61102c53 because the tablet was not found. No further retry: Not found: Tablet not found: a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:20.616987 17143 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:20.617347 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet d22f0f9d7e6a46a08ca9f411cc2c658b because the tablet was not found. No further retry: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b
W20260923 17:31:20.618703 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 051f2e478a324df98647d90107d55533 because the tablet was not found. No further retry: Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:20.619477 17343 tablet_replica.cc:330] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:20.628055 17146 ts_tablet_manager.cc:1428] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.069s user 0.009s sys 0.000s
I20260923 17:31:20.629448 17146 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:20.630659 17600 consensus_queue.cc:1035] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:20.639637 17343 raft_consensus.cc:2229] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.641495 17343 raft_consensus.cc:2258] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.643910 16382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:20.645437 17343 ts_tablet_manager.cc:1905] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.650638 17143 tablet_bootstrap.cc:654] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.657541 17631 raft_consensus.cc:482] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.658268 17631 raft_consensus.cc:504] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.662326 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.663077 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:20.663988 16972 leader_election.cc:341] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.665925 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:20.667435 16972 leader_election.cc:341] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.667765 16382 catalog_manager.cc:6565] ReplaceTablet: tablet e818f6192de6472cb27747c0e3a1f128 deleted and replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c
I20260923 17:31:20.667850 16972 leader_election.cc:302] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:20.668656 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:20.669555 17631 leader_election.cc:288] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:20.669754 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:20.670380 17631 raft_consensus.cc:2726] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.672830 17403 ts_tablet_manager.cc:1918] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.673205 17403 log.cc:1196] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:20.676306 17144 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.685154 17144 ts_tablet_manager.cc:1397] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.091s user 0.015s sys 0.005s
I20260923 17:31:20.689177 17146 tablet_bootstrap.cc:654] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.689376 17603 consensus_queue.cc:1035] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:20.690088 17403 ts_tablet_manager.cc:1939] T 3b9b734b250d4920abbafe76f52ec540 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:20.692822 17603 raft_consensus.cc:482] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.693264 17603 raft_consensus.cc:504] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.696350 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ccdf1bd7a64f51965e0a6dc7e033c9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:20.697808 16677 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:20.701062 17603 leader_election.cc:288] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:20.703692 16678 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:20.704145 16678 leader_election.cc:302] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:20.705754 17144 raft_consensus.cc:348] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.706766 17143 tablet_bootstrap.cc:492] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:20.707207 17143 ts_tablet_manager.cc:1397] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.090s user 0.009s sys 0.000s
I20260923 17:31:20.708537 17144 raft_consensus.cc:374] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.710418 17144 raft_consensus.cc:729] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.706571 17603 raft_consensus.cc:2726] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:20.706105 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 132) took 1723 ms. Trace:
I20260923 17:31:20.714439 17403 rpcz_store.cc:276] 0923 17:31:18.982327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.982471 (+ 144us) service_pool.cc:227] Handling call
0923 17:31:20.309032 (+1326561us) ts_tablet_manager.cc:1145] Deleting tablet 3b9b734b250d4920abbafe76f52ec540
0923 17:31:20.309075 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.578345 (+269270us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.672578 (+ 94233us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.706038 (+ 33460us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1326030}
I20260923 17:31:20.718267 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 3b9b734b250d4920abbafe76f52ec540 (table default.Table_f03a6d12e8cb4f5298b3da28260ac01f [id=b7f11ec3dfc747da9682989860df9641]) successfully deleted
I20260923 17:31:20.719079 17403 tablet_replica.cc:330] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:20.721668 17146 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:20.722117 17146 ts_tablet_manager.cc:1397] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.093s user 0.007s sys 0.002s
I20260923 17:31:20.717113 17144 consensus_queue.cc:260] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.723330 17403 raft_consensus.cc:2229] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.727753 17403 raft_consensus.cc:2258] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.726699 17144 ts_tablet_manager.cc:1428] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260923 17:31:20.737742 17144 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:20.709703 17143 raft_consensus.cc:348] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.741976 17143 raft_consensus.cc:374] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.742523 17143 raft_consensus.cc:729] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.743139 17143 consensus_queue.cc:260] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.745252 17143 ts_tablet_manager.cc:1428] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260923 17:31:20.746120 17143 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:20.740988 17403 ts_tablet_manager.cc:1905] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.752583 17542 ts_tablet_manager.cc:1918] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.752966 17542 log.cc:1196] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:20.753661 17542 ts_tablet_manager.cc:1939] T ff22ffe24d424b1e8659579febfe1270 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:20.750756 17144 tablet_bootstrap.cc:654] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.776707 17343 ts_tablet_manager.cc:1918] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.773578 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 135) took 1767 ms. Trace:
I20260923 17:31:20.777421 17542 rpcz_store.cc:276] 0923 17:31:19.005671 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.005754 (+ 83us) service_pool.cc:227] Handling call
0923 17:31:20.545251 (+1539497us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:20.545303 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.643359 (+ 98056us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.752089 (+108730us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.773500 (+ 21411us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1539033}
I20260923 17:31:20.778693 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet ff22ffe24d424b1e8659579febfe1270 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
I20260923 17:31:20.779690 17542 tablet_replica.cc:330] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:20.780608 17542 raft_consensus.cc:2229] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.783658 17343 log.cc:1196] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:20.784371 17343 ts_tablet_manager.cc:1939] T a80034cbf6354e1bbe0a0220e43251c3 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:20.781493 17571 raft_consensus.cc:482] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.784826 17571 raft_consensus.cc:504] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.786610 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a80034cbf6354e1bbe0a0220e43251c3 (table default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]) successfully deleted
I20260923 17:31:20.786770 17571 leader_election.cc:288] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.789101 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c842eff86634bf4bc70c426b5226d66" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.789757 16743 raft_consensus.cc:2454] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
W20260923 17:31:20.790812 16827 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:20.791807 16826 leader_election.cc:302] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:20.792858 17571 raft_consensus.cc:2781] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:20.793215 17571 raft_consensus.cc:482] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260923 17:31:20.793447 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 261d1d03290546969a74ca60beb890eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 261d1d03290546969a74ca60beb890eb already in progress: creating tablet
I20260923 17:31:20.793608 17571 raft_consensus.cc:3037] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.798532 17603 consensus_queue.cc:1035] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:20.801041 17343 tablet_replica.cc:330] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:20.802919 17343 raft_consensus.cc:2229] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.804725 17542 raft_consensus.cc:2258] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.804953 17343 raft_consensus.cc:2258] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.806972 17144 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.807392 17144 ts_tablet_manager.cc:1397] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.070s user 0.007s sys 0.002s
I20260923 17:31:20.808753 16385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 42ccd82f191942239ffb88459d362107
I20260923 17:31:20.810101 16385 catalog_manager.cc:6565] ReplaceTablet: tablet 42ccd82f191942239ffb88459d362107 deleted and replaced by tablet 8a2f1143dcd64b2dbd25e17787096903
I20260923 17:31:20.810806 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:20.811280 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:20.812489 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_7eeb705b716645669b9ce77150103ab9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:20.809861 17343 ts_tablet_manager.cc:1905] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.817152 17144 raft_consensus.cc:348] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.817834 17144 raft_consensus.cc:374] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.818077 17144 raft_consensus.cc:729] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:20.819355 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eeb705b716645669b9ce77150103ab9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.823454 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:20.831053 17534 raft_consensus.cc:482] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.835405 17534 raft_consensus.cc:504] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.837381 17534 leader_election.cc:288] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.839329 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:20.840052 17041 raft_consensus.cc:2454] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:20.841374 16827 leader_election.cc:302] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:20.842274 17534 raft_consensus.cc:2781] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:20.843176 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:20.843845 16743 raft_consensus.cc:2454] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:20.842648 17534 raft_consensus.cc:482] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:20.844933 17534 raft_consensus.cc:3037] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.846814 17146 raft_consensus.cc:348] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.847536 17146 raft_consensus.cc:374] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.847785 17146 raft_consensus.cc:729] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:20.855195 17534 raft_consensus.cc:504] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.818645 17144 consensus_queue.cc:260] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.861032 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:20.862500 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:20.862845 17603 raft_consensus.cc:482] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:20.863272 17603 raft_consensus.cc:504] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.864485 17144 ts_tablet_manager.cc:1428] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.057s user 0.005s sys 0.002s
I20260923 17:31:20.864511 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet cbb9840f8b6d464caae440fb03a7be54 (table default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3]). Expected version 1 got 0
I20260923 17:31:20.868443 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:20.869081 17041 raft_consensus.cc:2454] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:20.869094 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_9621239dc2864f8798ad1057b67f76c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:20.866459 17146 consensus_queue.cc:260] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.870385 16678 leader_election.cc:302] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:20.871328 17603 leader_election.cc:288] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.872953 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5b5504f7174bf483e9ddc79161079f (DEFAULT_TABLE table=default.Table_7bc6c27fedc843e68f19f6075fdb755b [id=a4677c024a464676b028a4ff1081733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:20.874009 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5b5504f7174bf483e9ddc79161079f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:20.871943 17603 raft_consensus.cc:2781] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:20.875535 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9621239dc2864f8798ad1057b67f76c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:20.860152 17403 ts_tablet_manager.cc:1918] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:20.876451 17603 raft_consensus.cc:482] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:20.876763 17403 log.cc:1196] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/723000dc175844e990740dc503b53ced
I20260923 17:31:20.870481 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:20.877189 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5b5504f7174bf483e9ddc79161079f (DEFAULT_TABLE table=default.Table_7bc6c27fedc843e68f19f6075fdb755b [id=a4677c024a464676b028a4ff1081733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:20.877555 17403 ts_tablet_manager.cc:1939] T 723000dc175844e990740dc503b53ced P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:20.878517 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5b5504f7174bf483e9ddc79161079f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:20.879352 17146 ts_tablet_manager.cc:1428] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.157s user 0.008s sys 0.000s
I20260923 17:31:20.876928 17603 raft_consensus.cc:3037] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:20.878237 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 2 got 0
I20260923 17:31:20.879842 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 133) took 1896 ms. Trace:
I20260923 17:31:20.880620 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:20.888450 17146 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:20.891346 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 (DEFAULT_TABLE table=default.Table_841cfa422dfb457eb8f606f44dfa4c7c [id=aaedb0c6cff242f2acaf82970a6c0e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:20.893595 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c929b8beaaf4c7aa668852b63af2204. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:20.896754 17642 log.cc:924] Time spent T a3ee9185751d49e89e1c5c85ddab1661 P e5c071cdf65042a59cad226a15a5532c: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260923 17:31:20.880141 17403 rpcz_store.cc:276] 0923 17:31:18.983635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:18.983759 (+ 124us) service_pool.cc:227] Handling call
0923 17:31:20.718503 (+1734744us) ts_tablet_manager.cc:1145] Deleting tablet 723000dc175844e990740dc503b53ced
0923 17:31:20.718546 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.781041 (+ 62495us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.859941 (+ 78900us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:20.879781 (+ 19840us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1734262}
I20260923 17:31:20.908667 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 723000dc175844e990740dc503b53ced (table default.Table_c6ffc66fd1a34a139ff3848a8e5effc6 [id=d89380c569d9441f9a6bdecf55bda74e]) successfully deleted
I20260923 17:31:20.909881 17403 tablet_replica.cc:330] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:20.911794 17403 raft_consensus.cc:2229] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:20.920864 17144 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:20.922592 17403 raft_consensus.cc:2258] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:20.931708 17600 consensus_queue.cc:1035] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:20.932190 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 006895667f324d049debbebb132ed09b
I20260923 17:31:20.933705 16381 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 177) took 1599 ms. Trace:
I20260923 17:31:20.937237 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:20.937497 16721 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:20.934015 16381 rpcz_store.cc:276] 0923 17:31:19.334483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.429220 (+ 94737us) service_pool.cc:227] Handling call
0923 17:31:19.583950 (+154730us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df466 00007f9127120b34 00007f9127120ac2 00007f91271209f1 00007f912712096c 00007f912711fc22 00007f91253be94c 00007f91253bdd86
0923 17:31:19.594553 (+ 10603us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:19.608686 (+ 14133us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:19.608836 (+ 150us) catalog_manager.cc:2761] Locking tablets
0923 17:31:19.690621 (+ 81785us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:20.029310 (+338689us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:20.029543 (+ 233us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:20.932069 (+902526us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:20.933644 (+ 1575us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:19.859459 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:19.860487 (+ 1028us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:19.860511 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:19.860524 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:19.860852 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:19.860894 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:19.860916 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:19.861192 (+ 276us) tablet.cc:699] Row locks acquired
0923 17:31:19.861205 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:19.861314 (+ 109us) write_op.cc:270] Start()
0923 17:31:19.861397 (+ 83us) write_op.cc:276] Timestamp: P: 1790184679861269 usec, L: 0
0923 17:31:19.861412 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:19.861878 (+ 466us) log.cc:842] Serialized 734 byte log entry
0923 17:31:20.020800 (+158922us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.022624 (+ 1824us) write_op.cc:301] APPLY: starting
0923 17:31:20.022686 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.022803 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.022895 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.023141 (+ 246us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.023291 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:20.023340 (+ 49us) write_op.cc:312] APPLY: finished
0923 17:31:20.023576 (+ 236us) log.cc:842] Serialized 29 byte log entry
0923 17:31:20.023736 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.023814 (+ 78us) write_op.cc:454] Released schema lock
0923 17:31:20.023849 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:20.024038 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.024063 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":353371904,"child_traces":[["op",{"apply.queue_time_us":1650,"apply.run_cpu_time_us":1632,"apply.run_wall_time_us":1631,"num_ops":2,"prepare.queue_time_us":161612,"prepare.run_cpu_time_us":2017,"prepare.run_wall_time_us":98800,"replication_time_us":159259}]]}
I20260923 17:31:20.831367 17542 ts_tablet_manager.cc:1905] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.940653 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2" } modify_external_catalogs: true
I20260923 17:31:20.941264 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2" } modify_external_catalogs: true
W20260923 17:31:20.943361 16677 consensus_peers.cc:487] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:20.945582 17403 ts_tablet_manager.cc:1905] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:20.951705 17144 tablet_bootstrap.cc:654] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:20.951649 16379 catalog_manager.cc:5495] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:20.953167 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:20.954042 17600 consensus_queue.cc:1035] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:20.826195 17571 raft_consensus.cc:504] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.865703 16867 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5b5504f7174bf483e9ddc79161079f (DEFAULT_TABLE table=default.Table_7bc6c27fedc843e68f19f6075fdb755b [id=a4677c024a464676b028a4ff1081733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:20.971807 16867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5b5504f7174bf483e9ddc79161079f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:20.883966 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 (DEFAULT_TABLE table=default.Table_841cfa422dfb457eb8f606f44dfa4c7c [id=aaedb0c6cff242f2acaf82970a6c0e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:20.901057 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:20.937812 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:20.976949 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c929b8beaaf4c7aa668852b63af2204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:20.990234 17571 leader_election.cc:288] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:20.996183 17144 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:20.996644 17144 ts_tablet_manager.cc:1397] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.001s
I20260923 17:31:20.998181 16379 catalog_manager.cc:5495] T a3ee9185751d49e89e1c5c85ddab1661 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:20.998852 17144 raft_consensus.cc:348] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:20.999440 17144 raft_consensus.cc:374] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:20.999647 17144 raft_consensus.cc:729] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.000154 17144 consensus_queue.cc:260] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.005599 17603 raft_consensus.cc:504] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.009178 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 (DEFAULT_TABLE table=default.Table_841cfa422dfb457eb8f606f44dfa4c7c [id=aaedb0c6cff242f2acaf82970a6c0e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.010463 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c929b8beaaf4c7aa668852b63af2204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.015503 16721 tablet_service.cc:1467] Processing CreateTablet for tablet e09c716342a6445a8edab130fd78a8e3 (DEFAULT_TABLE table=default.Table_b4fa8697648e41faba9af136a0f73a3f [id=2c5ce77a4e2542dcb66c26c0a38a9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.016445 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09c716342a6445a8edab130fd78a8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.018253 17021 tablet_service.cc:1467] Processing CreateTablet for tablet e09c716342a6445a8edab130fd78a8e3 (DEFAULT_TABLE table=default.Table_b4fa8697648e41faba9af136a0f73a3f [id=2c5ce77a4e2542dcb66c26c0a38a9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.019164 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09c716342a6445a8edab130fd78a8e3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:20.991920 16827 leader_election.cc:341] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:21.000837 17143 tablet_bootstrap.cc:654] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.023452 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:21.024045 17040 raft_consensus.cc:3037] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.029645 17146 tablet_bootstrap.cc:654] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.030898 17040 raft_consensus.cc:2454] T 3db3bfd8b67a4290a007de5b4b9d0e2e P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:21.031991 16827 leader_election.cc:302] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:21.036959 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_219bd5224fe64024867dec9b8bef6824"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:21.038592 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_219bd5224fe64024867dec9b8bef6824 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:21.039101 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c842eff86634bf4bc70c426b5226d66" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:21.039389 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db3bfd8b67a4290a007de5b4b9d0e2e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
W20260923 17:31:21.039582 16677 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.039696 16743 raft_consensus.cc:3037] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.036060 17571 raft_consensus.cc:2781] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:21.039983 16744 raft_consensus.cc:3037] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.040491 17571 raft_consensus.cc:686] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:21.041306 17571 consensus_queue.cc:237] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.002543 17534 leader_election.cc:288] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.010885 16873 tablet_service.cc:1467] Processing CreateTablet for tablet e09c716342a6445a8edab130fd78a8e3 (DEFAULT_TABLE table=default.Table_b4fa8697648e41faba9af136a0f73a3f [id=2c5ce77a4e2542dcb66c26c0a38a9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.044477 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09c716342a6445a8edab130fd78a8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.047431 16744 raft_consensus.cc:2454] T 3db3bfd8b67a4290a007de5b4b9d0e2e P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:21.047581 16743 raft_consensus.cc:2454] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:21.014729 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:20.886512 17343 ts_tablet_manager.cc:1918] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:21.055322 17343 log.cc:1196] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/e818f6192de6472cb27747c0e3a1f128
W20260923 17:31:21.060112 16677 leader_election.cc:341] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.061651 16826 leader_election.cc:302] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:21.063210 17643 raft_consensus.cc:2781] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:21.063671 17643 raft_consensus.cc:686] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:21.063153 17644 consensus_queue.cc:1035] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260923 17:31:21.064467 17643 consensus_queue.cc:237] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.075309 16871 tablet_service.cc:1467] Processing CreateTablet for tablet a2e5bc4ae186404c896814b95d6cc7ef (DEFAULT_TABLE table=default.Table_7e13d4e4841d47c18ecc2442c83ec934 [id=0160315adde14078a4216959ea769003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.076367 16725 tablet_service.cc:1467] Processing CreateTablet for tablet a2e5bc4ae186404c896814b95d6cc7ef (DEFAULT_TABLE table=default.Table_7e13d4e4841d47c18ecc2442c83ec934 [id=0160315adde14078a4216959ea769003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.076895 16871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2e5bc4ae186404c896814b95d6cc7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.077387 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2e5bc4ae186404c896814b95d6cc7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.056094 17343 ts_tablet_manager.cc:1939] T e818f6192de6472cb27747c0e3a1f128 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:21.073715 16379 catalog_manager.cc:5495] T 6dbfc01e66404a5da4f30c5eff8e7f45 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:21.096304 17343 tablet_replica.cc:330] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:21.100497 17403 ts_tablet_manager.cc:1918] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:21.101512 17403 log.cc:1196] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:21.102334 17403 ts_tablet_manager.cc:1939] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:21.108688 16866 tablet_service.cc:1467] Processing CreateTablet for tablet adc1f188f61e40a2ae42fcb716f5619c (DEFAULT_TABLE table=default.Table_cfb90e755e9d41a5bf5d6636cecfb5a6 [id=816bb4193c0446f2808739f6fd38ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.109825 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1f188f61e40a2ae42fcb716f5619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.110103 17146 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:21.111783 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
I20260923 17:31:21.112555 17146 ts_tablet_manager.cc:1397] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.224s user 0.009s sys 0.001s
I20260923 17:31:21.121341 17021 tablet_service.cc:1467] Processing CreateTablet for tablet a2e5bc4ae186404c896814b95d6cc7ef (DEFAULT_TABLE table=default.Table_7e13d4e4841d47c18ecc2442c83ec934 [id=0160315adde14078a4216959ea769003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.122396 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2e5bc4ae186404c896814b95d6cc7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.107486 17650 consensus_queue.cc:1035] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:21.124056 17528 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.126829 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 135) took 1997 ms. Trace:
I20260923 17:31:21.126644 17528 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.127214 17403 rpcz_store.cc:276] 0923 17:31:19.129137 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.129265 (+ 128us) service_pool.cc:227] Handling call
0923 17:31:20.909738 (+1780473us) ts_tablet_manager.cc:1145] Deleting tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d
0923 17:31:20.909782 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:20.975591 (+ 65809us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.100197 (+124606us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.126759 (+ 26562us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6864,"tablet-delete.queue_time_us":1779857}
I20260923 17:31:21.128665 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (table default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]) successfully deleted
I20260923 17:31:21.128443 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 136) took 1907 ms. Trace:
I20260923 17:31:21.129349 17403 rpcz_store.cc:276] 0923 17:31:19.221102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.221266 (+ 164us) service_pool.cc:227] Handling call
0923 17:31:21.128173 (+1906907us) ts_tablet_manager.cc:1145] Deleting tablet d22f0f9d7e6a46a08ca9f411cc2c658b
0923 17:31:21.128212 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.128391 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1905984}
W20260923 17:31:21.130425 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet d22f0f9d7e6a46a08ca9f411cc2c658b because the tablet was not found. No further retry: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:21.131431 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e99ac8c0549b188cf0d872f33d1bf (DEFAULT_TABLE table=default.Table_93989af63763474587fe6c96b168a123 [id=5482ef9413e743d49623fb25e8767e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.132640 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e99ac8c0549b188cf0d872f33d1bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.097116 17343 raft_consensus.cc:2229] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:21.133737 17018 tablet_service.cc:1467] Processing CreateTablet for tablet adc1f188f61e40a2ae42fcb716f5619c (DEFAULT_TABLE table=default.Table_cfb90e755e9d41a5bf5d6636cecfb5a6 [id=816bb4193c0446f2808739f6fd38ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.134150 17343 raft_consensus.cc:2258] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:21.134647 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1f188f61e40a2ae42fcb716f5619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.135079 17143 tablet_bootstrap.cc:492] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:21.135552 17143 ts_tablet_manager.cc:1397] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.390s user 0.008s sys 0.002s
I20260923 17:31:21.136054 17343 ts_tablet_manager.cc:1905] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:21.137089 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 107a073567314be996d6b8e3dbc47701 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.137040 17013 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e99ac8c0549b188cf0d872f33d1bf (DEFAULT_TABLE table=default.Table_93989af63763474587fe6c96b168a123 [id=5482ef9413e743d49623fb25e8767e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.138046 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107a073567314be996d6b8e3dbc47701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.138084 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e99ac8c0549b188cf0d872f33d1bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.138239 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:21.138813 17039 raft_consensus.cc:3037] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.146920 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:21.132963 16718 tablet_service.cc:1467] Processing CreateTablet for tablet 107a073567314be996d6b8e3dbc47701 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.150774 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107a073567314be996d6b8e3dbc47701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.158764 17650 raft_consensus.cc:482] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.159257 17650 raft_consensus.cc:504] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.161267 17650 leader_election.cc:288] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:21.163504 16678 leader_election.cc:341] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:21.165169 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a60bf118082b4e9991bfeb3ac59eb118" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:21.129444 17144 ts_tablet_manager.cc:1428] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.132s user 0.006s sys 0.004s
I20260923 17:31:21.168608 17014 tablet_service.cc:1467] Processing CreateTablet for tablet 107a073567314be996d6b8e3dbc47701 (DEFAULT_TABLE table=default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.169010 16868 tablet_service.cc:1467] Processing CreateTablet for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (DEFAULT_TABLE table=default.Table_c1fb4b15a00641eda2af13db76fd5ec6 [id=3acfda57412f48e4a9561d2b4cf7f98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.169567 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107a073567314be996d6b8e3dbc47701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.169653 16725 tablet_service.cc:1467] Processing CreateTablet for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (DEFAULT_TABLE table=default.Table_c1fb4b15a00641eda2af13db76fd5ec6 [id=3acfda57412f48e4a9561d2b4cf7f98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.170653 16725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.170470 17039 raft_consensus.cc:2454] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:21.171763 17144 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:21.172613 17542 ts_tablet_manager.cc:1918] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:21.172999 17542 log.cc:1196] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:21.173645 17542 ts_tablet_manager.cc:1939] T a6f62eadc6a94aedb4a8164c545cda17 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:21.175411 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.176393 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (DEFAULT_TABLE table=default.Table_c1fb4b15a00641eda2af13db76fd5ec6 [id=3acfda57412f48e4a9561d2b4cf7f98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.176146 16379 catalog_manager.cc:5495] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:21.177439 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.177435 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:21.178021 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:21.179137 16379 catalog_manager.cc:5495] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
W20260923 17:31:21.181342 16677 leader_election.cc:341] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.184383 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 31a83c7168a64b64bd224ef5a4c6939e (DEFAULT_TABLE table=default.Table_1facf5dbeaf74ac2a94bf7f3d1aa7b1f [id=ef27e45755e6402b94b76403d372ccc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.185452 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31a83c7168a64b64bd224ef5a4c6939e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.185350 17143 raft_consensus.cc:348] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.186129 17143 raft_consensus.cc:374] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.186367 17143 raft_consensus.cc:729] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.186980 17143 consensus_queue.cc:260] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.184554 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 31a83c7168a64b64bd224ef5a4c6939e (DEFAULT_TABLE table=default.Table_1facf5dbeaf74ac2a94bf7f3d1aa7b1f [id=ef27e45755e6402b94b76403d372ccc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.189275 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31a83c7168a64b64bd224ef5a4c6939e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:21.194878 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e329dd46a3a245b6826b01ac8de56621 consensus state has no leader
I20260923 17:31:21.195107 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 31a83c7168a64b64bd224ef5a4c6939e (DEFAULT_TABLE table=default.Table_1facf5dbeaf74ac2a94bf7f3d1aa7b1f [id=ef27e45755e6402b94b76403d372ccc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.196210 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31a83c7168a64b64bd224ef5a4c6939e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.173760 16678 leader_election.cc:302] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:21.200757 17650 raft_consensus.cc:2781] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:21.201224 17650 raft_consensus.cc:686] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:21.202229 17650 consensus_queue.cc:237] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.204408 16677 leader_election.cc:302] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:21.205487 17651 raft_consensus.cc:2726] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:21.209447 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 006895667f324d049debbebb132ed09b (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.210417 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006895667f324d049debbebb132ed09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.213721 17012 tablet_service.cc:1467] Processing CreateTablet for tablet 006895667f324d049debbebb132ed09b (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.214974 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006895667f324d049debbebb132ed09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.230675 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7c8e360d954ce78913e78c8662c341 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.231751 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7c8e360d954ce78913e78c8662c341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.240245 17146 raft_consensus.cc:348] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.241058 17146 raft_consensus.cc:374] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.241348 17146 raft_consensus.cc:729] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.243275 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_67ae78aa612d4ebcbaf1dcb60c98be71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:21.248322 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 006895667f324d049debbebb132ed09b (DEFAULT_TABLE table=default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.248942 16872 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7c8e360d954ce78913e78c8662c341 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.249423 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 006895667f324d049debbebb132ed09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.249933 16872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7c8e360d954ce78913e78c8662c341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.250522 17403 tablet_replica.cc:330] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:21.251462 17403 raft_consensus.cc:2229] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:21.252000 17403 raft_consensus.cc:2258] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:21.253720 17403 ts_tablet_manager.cc:1905] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:21.261219 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:21.261998 17040 raft_consensus.cc:2454] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:21.269127 16726 tablet_service.cc:1467] Processing CreateTablet for tablet adc1f188f61e40a2ae42fcb716f5619c (DEFAULT_TABLE table=default.Table_cfb90e755e9d41a5bf5d6636cecfb5a6 [id=816bb4193c0446f2808739f6fd38ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.270788 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1f188f61e40a2ae42fcb716f5619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.276698 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 137) took 2049 ms. Trace:
I20260923 17:31:21.277055 17542 rpcz_store.cc:276] 0923 17:31:19.226958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.227089 (+ 131us) service_pool.cc:227] Handling call
0923 17:31:20.779522 (+1552433us) ts_tablet_manager.cc:1145] Deleting tablet a6f62eadc6a94aedb4a8164c545cda17
0923 17:31:20.779567 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.068515 (+288948us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.172358 (+103843us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.276634 (+104276us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1551874}
I20260923 17:31:21.278218 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 136) took 2125 ms. Trace:
I20260923 17:31:21.278501 17542 rpcz_store.cc:276] 0923 17:31:19.152726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.305405 (+152679us) service_pool.cc:227] Handling call
0923 17:31:21.277887 (+1972482us) ts_tablet_manager.cc:1145] Deleting tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d
0923 17:31:21.277926 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.278168 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1971505}
I20260923 17:31:21.279353 17542 tablet_replica.cc:330] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:21.279709 16678 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:21.280747 17146 consensus_queue.cc:260] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.283008 17146 ts_tablet_manager.cc:1428] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.170s user 0.005s sys 0.001s
I20260923 17:31:21.283131 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:21.283919 17146 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:21.280071 17542 raft_consensus.cc:2229] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:21.284936 17542 raft_consensus.cc:2258] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:21.286309 17143 ts_tablet_manager.cc:1428] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.150s user 0.004s sys 0.004s
I20260923 17:31:21.286775 17542 ts_tablet_manager.cc:1905] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:21.287164 17143 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:21.290345 17600 raft_consensus.cc:2781] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:21.290859 17600 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:21.291220 17600 raft_consensus.cc:3037] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.298836 17651 consensus_queue.cc:1035] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:21.301740 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e99ac8c0549b188cf0d872f33d1bf (DEFAULT_TABLE table=default.Table_93989af63763474587fe6c96b168a123 [id=5482ef9413e743d49623fb25e8767e49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.302778 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e99ac8c0549b188cf0d872f33d1bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:21.311146 16677 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.311715 17144 tablet_bootstrap.cc:654] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.310405 17600 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.313772 17600 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:21.312011 16677 consensus_peers.cc:487] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:21.321403 17343 ts_tablet_manager.cc:1918] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:21.321810 17343 log.cc:1196] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/42ccd82f191942239ffb88459d362107
I20260923 17:31:21.136869 17603 leader_election.cc:288] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.229777 17528 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.281234 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a6f62eadc6a94aedb4a8164c545cda17 (table default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]) successfully deleted
W20260923 17:31:21.326251 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 45041us
W20260923 17:31:21.327241 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d already in progress: creating tablet
W20260923 17:31:21.329027 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67ae78aa612d4ebcbaf1dcb60c98be71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:21.336720 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:21.338409 17039 raft_consensus.cc:3037] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.340171 17144 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:21.340595 17144 ts_tablet_manager.cc:1397] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.169s user 0.006s sys 0.004s
I20260923 17:31:21.342597 17144 raft_consensus.cc:348] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.343215 17144 raft_consensus.cc:374] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.343458 17144 raft_consensus.cc:729] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.346294 17039 raft_consensus.cc:2454] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:21.344017 17144 consensus_queue.cc:260] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.356012 17144 ts_tablet_manager.cc:1428] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.015s user 0.003s sys 0.002s
I20260923 17:31:21.357000 17144 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:21.350997 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7c8e360d954ce78913e78c8662c341 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:21.365335 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7c8e360d954ce78913e78c8662c341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:21.379069 17144 tablet_bootstrap.cc:654] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.374639 17343 ts_tablet_manager.cc:1939] T 42ccd82f191942239ffb88459d362107 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:21.406129 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 42ccd82f191942239ffb88459d362107 (table default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]) successfully deleted
W20260923 17:31:21.406869 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 006895667f324d049debbebb132ed09b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 006895667f324d049debbebb132ed09b already in progress: creating tablet
I20260923 17:31:21.412070 17146 tablet_bootstrap.cc:654] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.415251 17143 tablet_bootstrap.cc:700] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Time spent opening tablet: real 0.124s user 0.004s sys 0.000s
I20260923 17:31:21.415882 17143 tablet_bootstrap.cc:654] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.422025 17144 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:21.422410 17144 ts_tablet_manager.cc:1397] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.066s user 0.006s sys 0.004s
W20260923 17:31:21.438316 16827 consensus_peers.cc:487] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e818f6192de6472cb27747c0e3a1f128. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:21.439374 16827 consensus_peers.cc:487] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a80034cbf6354e1bbe0a0220e43251c3. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:21.436923 17144 raft_consensus.cc:348] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.441247 17144 raft_consensus.cc:374] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.441807 17144 raft_consensus.cc:729] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.442689 17144 consensus_queue.cc:260] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.450768 17144 ts_tablet_manager.cc:1428] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260923 17:31:21.451711 17144 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:21.460878 17144 tablet_bootstrap.cc:654] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.468374 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:21.473575 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:21.476354 17146 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:21.476938 17146 ts_tablet_manager.cc:1397] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.193s user 0.011s sys 0.000s
I20260923 17:31:21.479597 17146 raft_consensus.cc:348] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.480352 17146 raft_consensus.cc:374] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.480856 17146 raft_consensus.cc:729] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.481556 17146 consensus_queue.cc:260] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.483832 17146 ts_tablet_manager.cc:1428] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260923 17:31:21.485200 16388 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:21.490523 17403 ts_tablet_manager.cc:1918] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:21.490944 17403 log.cc:1196] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:21.501821 17403 ts_tablet_manager.cc:1939] T a6f62eadc6a94aedb4a8164c545cda17 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:21.490691 17146 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:21.506201 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:21.506938 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:21.508208 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:21.508934 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:21.511359 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 137) took 2282 ms. Trace:
I20260923 17:31:21.511670 17403 rpcz_store.cc:276] 0923 17:31:19.229091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.229205 (+ 114us) service_pool.cc:227] Handling call
0923 17:31:21.250345 (+2021140us) ts_tablet_manager.cc:1145] Deleting tablet a6f62eadc6a94aedb4a8164c545cda17
0923 17:31:21.250395 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.440784 (+190389us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.490237 (+ 49453us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.511286 (+ 21049us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2020486}
I20260923 17:31:21.512801 17403 tablet_replica.cc:330] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:21.513512 17403 raft_consensus.cc:2229] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:21.536877 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:21.541134 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05a6f83314d433a9502e0c00ba8ad09" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:21.541918 17143 tablet_bootstrap.cc:492] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:21.542352 17143 ts_tablet_manager.cc:1397] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.255s user 0.007s sys 0.009s
I20260923 17:31:21.543241 17542 ts_tablet_manager.cc:1918] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:21.543650 17542 log.cc:1196] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:21.544451 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:21.547039 17571 raft_consensus.cc:482] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.547521 17571 raft_consensus.cc:504] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.549474 17571 leader_election.cc:288] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.552866 16386 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 179) took 2035 ms. Trace:
I20260923 17:31:21.553153 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:21.553175 16386 rpcz_store.cc:276] 0923 17:31:19.517742 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.665353 (+147611us) service_pool.cc:227] Handling call
0923 17:31:19.841227 (+175874us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df7d6 00007f9127125634 00007f91271255c2 00007f91271254f1 00007f912712546c 00007f9127124722 00007f91253be94c 00007f91253bdd86
0923 17:31:19.842025 (+ 798us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:19.842641 (+ 616us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:19.843244 (+ 603us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:19.843452 (+ 208us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:20.025875 (+182423us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:21.552791 (+1526916us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:19.958406 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:19.984683 (+ 26277us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:19.988192 (+ 3509us) write_op.cc:435] Acquired schema lock
0923 17:31:19.988251 (+ 59us) tablet.cc:660] Decoding operations
0923 17:31:19.988581 (+ 330us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:19.988622 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:19.988643 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:19.988828 (+ 185us) tablet.cc:699] Row locks acquired
0923 17:31:19.988841 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:19.988972 (+ 131us) write_op.cc:270] Start()
0923 17:31:19.989064 (+ 92us) write_op.cc:276] Timestamp: P: 1790184679988915 usec, L: 0
0923 17:31:19.989080 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:19.989631 (+ 551us) log.cc:842] Serialized 405 byte log entry
0923 17:31:20.022100 (+ 32469us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.024395 (+ 2295us) write_op.cc:301] APPLY: starting
0923 17:31:20.024447 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.024536 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.024632 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.024784 (+ 152us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.024917 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:20.024959 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:20.025116 (+ 157us) log.cc:842] Serialized 23 byte log entry
0923 17:31:20.025253 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.025318 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:20.025348 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:20.025520 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.025545 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":402360704,"child_traces":[["op",{"apply.queue_time_us":2157,"apply.run_cpu_time_us":1321,"apply.run_wall_time_us":1320,"num_ops":1,"prepare.queue_time_us":113364,"prepare.run_cpu_time_us":2207,"prepare.run_wall_time_us":31452,"replication_time_us":32936}]]}
I20260923 17:31:21.576503 17542 ts_tablet_manager.cc:1939] T eb2beb8360d54efeb2fa74c005d011d1 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:21.568260 17146 tablet_bootstrap.cc:654] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.586622 17631 raft_consensus.cc:482] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.587150 17631 raft_consensus.cc:504] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.589123 17631 leader_election.cc:288] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:21.596374 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:21.598066 16972 leader_election.cc:341] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:21.599390 17675 log.cc:924] Time spent T 105455d69eb64adc8513b1bfa97e6330 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260923 17:31:21.602130 17143 raft_consensus.cc:348] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.611335 17143 raft_consensus.cc:374] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260923 17:31:21.611109 16677 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:21.611846 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 49303.5us
I20260923 17:31:21.613168 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a6f62eadc6a94aedb4a8164c545cda17 (table default.Table_40b23e3d14dd4c1da5acb4c38e0e466f [id=b841395a9ee74b1389d6f12fbf5b8465]) successfully deleted
I20260923 17:31:21.616482 16678 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:21.632964 17143 raft_consensus.cc:729] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.635843 17146 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:21.636340 17146 ts_tablet_manager.cc:1397] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.146s user 0.010s sys 0.002s
I20260923 17:31:21.637090 17143 consensus_queue.cc:260] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.655206 17143 ts_tablet_manager.cc:1428] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.113s user 0.006s sys 0.001s
I20260923 17:31:21.659119 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ee9185751d49e89e1c5c85ddab1661" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:21.662787 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:21.663758 16745 raft_consensus.cc:2454] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:21.665110 17649 raft_consensus.cc:2781] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:21.665817 17649 raft_consensus.cc:686] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:21.513944 17403 raft_consensus.cc:2258] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:21.667272 17649 consensus_queue.cc:237] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.604794 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 141) took 2288 ms. Trace:
I20260923 17:31:21.670825 17542 rpcz_store.cc:276] 0923 17:31:19.315819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.315936 (+ 117us) service_pool.cc:227] Handling call
0923 17:31:21.279217 (+1963281us) ts_tablet_manager.cc:1145] Deleting tablet eb2beb8360d54efeb2fa74c005d011d1
0923 17:31:21.279253 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.431986 (+152733us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.542998 (+111012us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.604170 (+ 61172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1962674}
I20260923 17:31:21.672199 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet eb2beb8360d54efeb2fa74c005d011d1 (table default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]) successfully deleted
I20260923 17:31:21.672467 17144 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:21.673087 17144 ts_tablet_manager.cc:1397] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.222s user 0.008s sys 0.017s
I20260923 17:31:21.673890 17528 raft_consensus.cc:482] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.674471 17528 raft_consensus.cc:504] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.676201 17144 raft_consensus.cc:348] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.676844 17403 ts_tablet_manager.cc:1905] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:21.676975 17144 raft_consensus.cc:374] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.677299 17144 raft_consensus.cc:729] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.677904 17144 consensus_queue.cc:260] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.681164 17603 consensus_queue.cc:1035] T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:21.684537 16972 leader_election.cc:302] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:21.691366 16826 leader_election.cc:302] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:21.692963 17571 raft_consensus.cc:2726] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:21.694974 16893 raft_consensus.cc:3037] T a3ee9185751d49e89e1c5c85ddab1661 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.702495 17631 raft_consensus.cc:2781] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:21.702950 17631 raft_consensus.cc:482] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:21.703225 17631 raft_consensus.cc:3037] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.709121 17144 ts_tablet_manager.cc:1428] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.036s user 0.004s sys 0.002s
I20260923 17:31:21.710304 17146 raft_consensus.cc:348] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.711208 17144 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:21.711254 17146 raft_consensus.cc:374] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.711596 17146 raft_consensus.cc:729] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.715059 17631 raft_consensus.cc:504] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.718374 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:21.718411 17146 consensus_queue.cc:260] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:21.719862 16972 leader_election.cc:341] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.721114 17528 leader_election.cc:288] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.725965 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ccd82f191942239ffb88459d362107" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:21.726094 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:21.726694 16745 raft_consensus.cc:3037] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:21.733359 16677 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.740413 17146 ts_tablet_manager.cc:1428] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.075s user 0.009s sys 0.000s
I20260923 17:31:21.737214 17631 leader_election.cc:288] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:21.689388 17143 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:21.744297 17542 tablet_replica.cc:330] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:21.745088 17542 raft_consensus.cc:2229] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:21.746021 17542 raft_consensus.cc:2258] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:21.753376 17146 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:21.758487 17542 ts_tablet_manager.cc:1905] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:21.769641 17143 tablet_bootstrap.cc:654] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.774171 17685 raft_consensus.cc:482] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.774631 17685 raft_consensus.cc:504] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.776455 17685 leader_election.cc:288] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
W20260923 17:31:21.776897 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 134193us
I20260923 17:31:21.777587 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:21.777570 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:21.778803 16972 leader_election.cc:341] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:21.780762 16972 leader_election.cc:341] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:21.781164 16972 leader_election.cc:302] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:21.781621 17146 tablet_bootstrap.cc:654] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.782330 17685 raft_consensus.cc:2726] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:21.798693 17143 tablet_bootstrap.cc:492] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:21.799122 17143 ts_tablet_manager.cc:1397] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.110s user 0.008s sys 0.000s
I20260923 17:31:21.803066 17143 raft_consensus.cc:348] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.804884 17143 raft_consensus.cc:374] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.805114 17143 raft_consensus.cc:729] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.807811 17143 consensus_queue.cc:260] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:21.813781 16678 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
I20260923 17:31:21.814267 16678 leader_election.cc:302] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:21.819157 17144 tablet_bootstrap.cc:654] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.823882 17146 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:21.826872 17143 ts_tablet_manager.cc:1428] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260923 17:31:21.829180 17146 ts_tablet_manager.cc:1397] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.076s user 0.004s sys 0.012s
I20260923 17:31:21.833654 17664 raft_consensus.cc:482] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:21.834088 17664 raft_consensus.cc:504] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.836563 17664 leader_election.cc:288] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:21.835371 17146 raft_consensus.cc:348] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.837302 17146 raft_consensus.cc:374] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:21.837541 17146 raft_consensus.cc:729] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:21.838207 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:21.839016 17041 raft_consensus.cc:2379] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:21.838994 17146 consensus_queue.cc:260] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.845685 17146 ts_tablet_manager.cc:1428] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.016s user 0.002s sys 0.003s
I20260923 17:31:21.826164 17603 raft_consensus.cc:2726] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:21.846248 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20c9b997a024151a810ea015c20e18f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:21.847784 16826 leader_election.cc:302] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:21.848778 17571 raft_consensus.cc:3037] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:21.850514 17146 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:21.855882 17146 tablet_bootstrap.cc:654] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.855997 17571 raft_consensus.cc:2726] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:21.861172 17692 tablet.cc:1722] T 85eba2aac8d84defa8ce4081d447bc63 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:21.862819 17694 tablet.cc:1722] T 85eba2aac8d84defa8ce4081d447bc63 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:21.897323 16745 raft_consensus.cc:2454] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
W20260923 17:31:21.904560 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 67823.5us
I20260923 17:31:21.920953 17697 tablet.cc:1722] T 85eba2aac8d84defa8ce4081d447bc63 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:21.898941 16972 leader_election.cc:302] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:21.932014 17685 raft_consensus.cc:2781] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:21.932513 17685 raft_consensus.cc:686] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:21.933256 17685 consensus_queue.cc:237] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:21.827755 17143 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:21.955303 17143 tablet_bootstrap.cc:654] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:21.887635 17542 ts_tablet_manager.cc:1918] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:21.956382 17542 log.cc:1196] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:21.957098 17542 ts_tablet_manager.cc:1939] T 1a182b6ced72428e9638465f0cc4e7ed P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:21.958988 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 139) took 2648 ms. Trace:
I20260923 17:31:21.959261 17542 rpcz_store.cc:276] 0923 17:31:19.310934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.317881 (+ 6947us) service_pool.cc:227] Handling call
0923 17:31:21.689688 (+2371807us) ts_tablet_manager.cc:1145] Deleting tablet 1a182b6ced72428e9638465f0cc4e7ed
0923 17:31:21.694459 (+ 4771us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.790303 (+ 95844us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.887383 (+ 97080us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.958928 (+ 71545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2371030}
I20260923 17:31:21.960412 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 1a182b6ced72428e9638465f0cc4e7ed (table default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]) successfully deleted
W20260923 17:31:21.972381 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 43570.4us
I20260923 17:31:21.919593 17403 ts_tablet_manager.cc:1918] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:21.974128 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 145) took 2276 ms. Trace:
I20260923 17:31:21.979753 17542 rpcz_store.cc:276] 0923 17:31:19.697731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.700998 (+ 3267us) service_pool.cc:227] Handling call
0923 17:31:21.972648 (+2271650us) ts_tablet_manager.cc:1145] Deleting tablet 547fda6e7bcf474e9f73ba9a6659d34f
0923 17:31:21.972688 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.974062 (+ 1374us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2270921}
W20260923 17:31:21.981593 17695 log.cc:924] Time spent T 105455d69eb64adc8513b1bfa97e6330 P be93fda0843b44f5abc49e6f35c69217: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260923 17:31:21.977452 17403 log.cc:1196] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:21.993187 17403 ts_tablet_manager.cc:1939] T 389ecb50e0c34b4d92b5296bb17f4ab1 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:22.005218 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 140) took 2665 ms. Trace:
I20260923 17:31:22.005584 17403 rpcz_store.cc:276] 0923 17:31:19.339955 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.340064 (+ 109us) service_pool.cc:227] Handling call
0923 17:31:21.512653 (+2172589us) ts_tablet_manager.cc:1145] Deleting tablet 389ecb50e0c34b4d92b5296bb17f4ab1
0923 17:31:21.512695 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:21.815938 (+303243us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:21.916433 (+100495us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.005120 (+ 88687us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9097,"tablet-delete.queue_time_us":2171962}
I20260923 17:31:22.007339 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 139) took 2668 ms. Trace:
I20260923 17:31:22.007336 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 389ecb50e0c34b4d92b5296bb17f4ab1 (table default.Table_be66ae961c4342319e013d42905625d1 [id=4b6a9d0915114d0f9408a597c0f9f73c]) successfully deleted
I20260923 17:31:22.007660 17403 rpcz_store.cc:276] 0923 17:31:19.338907 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.344329 (+ 5422us) service_pool.cc:227] Handling call
0923 17:31:22.006948 (+2662619us) ts_tablet_manager.cc:1145] Deleting tablet 8d65cfb2eb3648faab5869bba69cab3f
0923 17:31:22.007037 (+ 89us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.007283 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2650366}
W20260923 17:31:22.008725 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 8d65cfb2eb3648faab5869bba69cab3f because the tablet was not found. No further retry: Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:22.011425 17146 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:22.011828 17146 ts_tablet_manager.cc:1397] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.162s user 0.027s sys 0.033s
I20260923 17:31:22.013460 17692 tablet.cc:1722] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:22.013742 17697 tablet.cc:1722] T d2796fcb72f5497a92208b40224739e9 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:22.018642 17146 raft_consensus.cc:348] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.019424 17146 raft_consensus.cc:374] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.019665 17146 raft_consensus.cc:729] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.020341 17146 consensus_queue.cc:260] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.022194 17146 ts_tablet_manager.cc:1428] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
I20260923 17:31:22.024179 17679 consensus_queue.cc:1035] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:22.032668 17542 tablet_replica.cc:330] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:22.033383 17542 raft_consensus.cc:2229] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:22.033962 17144 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:22.034271 17542 raft_consensus.cc:2258] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.034125 17603 consensus_queue.cc:1035] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:22.034945 17144 ts_tablet_manager.cc:1397] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.324s user 0.017s sys 0.016s
I20260923 17:31:22.035933 17542 ts_tablet_manager.cc:1905] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.035310 17403 tablet_replica.cc:330] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:22.037894 17403 raft_consensus.cc:2229] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:22.038996 17403 raft_consensus.cc:2258] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.042467 16869 tablet_service.cc:1467] Processing CreateTablet for tablet a84e322a4de24cbabf450fceb098d6e7 (DEFAULT_TABLE table=default.Table_4de36798f891433eb8cf23065562cfc2 [id=36caf250f5024edba1821ab5af3d3525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.043260 17403 ts_tablet_manager.cc:1905] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.043308 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:22.046190 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:22.048869 17011 tablet_service.cc:1467] Processing CreateTablet for tablet a84e322a4de24cbabf450fceb098d6e7 (DEFAULT_TABLE table=default.Table_4de36798f891433eb8cf23065562cfc2 [id=36caf250f5024edba1821ab5af3d3525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.049412 16726 tablet_service.cc:1467] Processing CreateTablet for tablet a84e322a4de24cbabf450fceb098d6e7 (DEFAULT_TABLE table=default.Table_4de36798f891433eb8cf23065562cfc2 [id=36caf250f5024edba1821ab5af3d3525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.050647 16866 tablet_service.cc:1467] Processing CreateTablet for tablet c1d71b22338241839b184e50d7fdd8f6 (DEFAULT_TABLE table=default.Table_0cad32e2a8eb4f078e9be43733d0b50d [id=43e8aa0950f94f6586f91535465999ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.051479 17143 tablet_bootstrap.cc:492] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:22.051956 17143 ts_tablet_manager.cc:1397] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.224s user 0.009s sys 0.000s
I20260923 17:31:22.055259 17012 tablet_service.cc:1467] Processing CreateTablet for tablet c1d71b22338241839b184e50d7fdd8f6 (DEFAULT_TABLE table=default.Table_0cad32e2a8eb4f078e9be43733d0b50d [id=43e8aa0950f94f6586f91535465999ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.056675 16718 tablet_service.cc:1467] Processing CreateTablet for tablet c1d71b22338241839b184e50d7fdd8f6 (DEFAULT_TABLE table=default.Table_0cad32e2a8eb4f078e9be43733d0b50d [id=43e8aa0950f94f6586f91535465999ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.057459 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84e322a4de24cbabf450fceb098d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.056573 17143 raft_consensus.cc:348] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.058059 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d71b22338241839b184e50d7fdd8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.058058 17143 raft_consensus.cc:374] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.058286 17143 raft_consensus.cc:729] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.060355 17143 consensus_queue.cc:260] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.061268 17144 raft_consensus.cc:348] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.061949 17144 raft_consensus.cc:374] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.062187 17144 raft_consensus.cc:729] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.065656 17144 consensus_queue.cc:260] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.066771 17143 ts_tablet_manager.cc:1428] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.014s user 0.004s sys 0.001s
I20260923 17:31:22.069994 17143 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:22.075368 17144 ts_tablet_manager.cc:1428] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260923 17:31:22.077176 17603 raft_consensus.cc:482] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.077625 17603 raft_consensus.cc:504] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.078114 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:22.079509 17603 leader_election.cc:288] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.083371 17144 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:22.089919 17685 raft_consensus.cc:482] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.090374 17685 raft_consensus.cc:504] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.092124 17685 leader_election.cc:288] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:22.095834 17146 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:22.101779 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d71b22338241839b184e50d7fdd8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.103629 17403 ts_tablet_manager.cc:1918] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:22.104059 17403 log.cc:1196] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:22.104668 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84e322a4de24cbabf450fceb098d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.104821 17403 ts_tablet_manager.cc:1939] T ff22ffe24d424b1e8659579febfe1270 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:22.109684 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 254793351e5245af9a4fa1642cad5e82 (DEFAULT_TABLE table=default.Table_e890a913f28f4a3b935f9b3ebfd75b9b [id=71912e3e9a964d9e8610c77ba8fe2210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.110606 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254793351e5245af9a4fa1642cad5e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.110440 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 254793351e5245af9a4fa1642cad5e82 (DEFAULT_TABLE table=default.Table_e890a913f28f4a3b935f9b3ebfd75b9b [id=71912e3e9a964d9e8610c77ba8fe2210]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:22.111080 16678 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:22.111487 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254793351e5245af9a4fa1642cad5e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.120985 17144 tablet_bootstrap.cc:654] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:22.123375 17631 raft_consensus.cc:482] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.079586 16385 catalog_manager.cc:5495] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:22.123797 17631 raft_consensus.cc:504] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.124527 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:22.125527 17631 leader_election.cc:288] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:22.125605 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet d2796fcb72f5497a92208b40224739e9 (table default.Table_632f010f9d114ecbab7901aa664556ac [id=17f994d2e127410685fffe47828a215b]). Expected version 1 got 0
I20260923 17:31:22.133587 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.137131 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.138296 16745 raft_consensus.cc:2454] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
W20260923 17:31:22.139628 16972 leader_election.cc:341] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.140180 16972 leader_election.cc:302] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:22.143362 17685 raft_consensus.cc:2781] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:22.143664 17685 raft_consensus.cc:482] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:22.143937 17685 raft_consensus.cc:3037] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.149384 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 143) took 2702 ms. Trace:
I20260923 17:31:22.150126 17631 raft_consensus.cc:482] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.149747 17403 rpcz_store.cc:276] 0923 17:31:19.446847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.446962 (+ 115us) service_pool.cc:227] Handling call
0923 17:31:22.017217 (+2570255us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:22.035193 (+ 17976us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.079826 (+ 44633us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.103373 (+ 23547us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.149320 (+ 45947us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2563500}
I20260923 17:31:22.150919 17403 tablet_replica.cc:330] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:22.150573 17631 raft_consensus.cc:504] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.151104 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet ff22ffe24d424b1e8659579febfe1270 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
I20260923 17:31:22.151749 17403 raft_consensus.cc:2229] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:22.152380 17631 leader_election.cc:288] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:22.152441 17403 raft_consensus.cc:2258] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.155072 17403 ts_tablet_manager.cc:1905] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.155226 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:22.156662 16381 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 193) took 1583 ms. Trace:
I20260923 17:31:22.157447 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.156956 16381 rpcz_store.cc:276] 0923 17:31:20.572912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.940094 (+367182us) service_pool.cc:227] Handling call
0923 17:31:20.941749 (+ 1655us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:20.942265 (+ 516us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:20.942384 (+ 119us) catalog_manager.cc:2761] Locking tablets
0923 17:31:20.944872 (+ 2488us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:20.996338 (+ 51466us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.996369 (+ 31us) write_op.cc:454] Released schema lock
0923 17:31:20.996636 (+ 267us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:20.996861 (+ 225us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:22.155120 (+1158259us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:22.156591 (+ 1471us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:20.946674 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:20.947060 (+ 386us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:20.947085 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:20.947098 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:20.947403 (+ 305us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:20.947446 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:20.947468 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:20.947756 (+ 288us) tablet.cc:699] Row locks acquired
0923 17:31:20.947770 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:20.947884 (+ 114us) write_op.cc:270] Start()
0923 17:31:20.947968 (+ 84us) write_op.cc:276] Timestamp: P: 1790184680947836 usec, L: 0
0923 17:31:20.947984 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:20.948497 (+ 513us) log.cc:842] Serialized 734 byte log entry
0923 17:31:20.958806 (+ 10309us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.959175 (+ 369us) write_op.cc:301] APPLY: starting
0923 17:31:20.959236 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.959377 (+ 141us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.959491 (+ 114us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.959848 (+ 357us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.960018 (+ 170us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:20.960074 (+ 56us) write_op.cc:312] APPLY: finished
0923 17:31:20.960315 (+ 241us) log.cc:842] Serialized 29 byte log entry
0923 17:31:20.960474 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.960556 (+ 82us) write_op.cc:454] Released schema lock
0923 17:31:20.960595 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":193,"apply.run_cpu_time_us":1618,"apply.run_wall_time_us":1618,"num_ops":2,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":2136,"prepare.run_wall_time_us":5210,"raft.queue_time_us":88,"raft.run_cpu_time_us":675,"raft.run_wall_time_us":676,"replication_time_us":10714}]]}
I20260923 17:31:22.160749 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:22.162392 16972 leader_election.cc:341] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.169814 17146 tablet_bootstrap.cc:654] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:22.170365 16718 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.082124 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051f2e478a324df98647d90107d55533" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.093894 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84e322a4de24cbabf450fceb098d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.179141 17144 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:22.179658 17144 ts_tablet_manager.cc:1397] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.096s user 0.011s sys 0.005s
W20260923 17:31:22.180569 16677 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.180989 16677 leader_election.cc:302] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:22.181990 17679 raft_consensus.cc:2726] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:22.093894 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.105227 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 254793351e5245af9a4fa1642cad5e82 (DEFAULT_TABLE table=default.Table_e890a913f28f4a3b935f9b3ebfd75b9b [id=71912e3e9a964d9e8610c77ba8fe2210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.183588 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254793351e5245af9a4fa1642cad5e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.127697 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.189203 17685 raft_consensus.cc:504] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.205839 17144 raft_consensus.cc:348] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.206521 17144 raft_consensus.cc:374] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.206770 17144 raft_consensus.cc:729] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.207319 17144 consensus_queue.cc:260] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.210845 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d71b22338241839b184e50d7fdd8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.127236 17705 tablet.cc:1722] T d2796fcb72f5497a92208b40224739e9 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:22.141036 17571 consensus_queue.cc:1035] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:22.217204 17571 consensus_queue.cc:1035] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:22.155002 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.155236 17664 raft_consensus.cc:482] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.219041 17664 raft_consensus.cc:504] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.220742 17664 leader_election.cc:288] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.162961 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.220904 17403 ts_tablet_manager.cc:1918] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:22.172557 16868 tablet_service.cc:1467] Processing CreateTablet for tablet ef890c51e92244f4b97c490f24250679 (DEFAULT_TABLE table=default.Table_7eeb705b716645669b9ce77150103ab9 [id=0b18a95235434180ac58aae4df832c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.221282 17403 log.cc:1196] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:22.221979 17403 ts_tablet_manager.cc:1939] T 1a182b6ced72428e9638465f0cc4e7ed P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:22.222041 16868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef890c51e92244f4b97c490f24250679. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:22.222319 16827 leader_election.cc:341] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:22.224073 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 144) took 2753 ms. Trace:
I20260923 17:31:22.224376 17403 rpcz_store.cc:276] 0923 17:31:19.470227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.470333 (+ 106us) service_pool.cc:227] Handling call
0923 17:31:22.150723 (+2680390us) ts_tablet_manager.cc:1145] Deleting tablet 1a182b6ced72428e9638465f0cc4e7ed
0923 17:31:22.150793 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.188148 (+ 37355us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.220659 (+ 32511us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.224018 (+ 3359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2679760}
I20260923 17:31:22.225558 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 1a182b6ced72428e9638465f0cc4e7ed (table default.Table_46a8dbb79853475b8593a2de25a1bc3b [id=5363fc3bd58241e190422cd3b6bbc6ff]) successfully deleted
I20260923 17:31:22.228842 17403 tablet_replica.cc:330] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:22.230499 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a60bf118082b4e9991bfeb3ac59eb118" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.231312 16743 raft_consensus.cc:2454] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:22.234481 16826 leader_election.cc:302] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:22.240108 17709 raft_consensus.cc:2781] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:22.241097 16972 leader_election.cc:341] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:22.242435 16972 leader_election.cc:341] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.242832 16972 leader_election.cc:302] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:22.244482 16972 leader_election.cc:341] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.244887 16972 leader_election.cc:302] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:22.253216 17709 raft_consensus.cc:482] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:22.254516 17709 raft_consensus.cc:3037] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:22.254606 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e329dd46a3a245b6826b01ac8de56621 consensus state has no leader
I20260923 17:31:22.266762 17664 consensus_queue.cc:1035] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:22.269064 17631 raft_consensus.cc:2726] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:22.270057 17631 raft_consensus.cc:2726] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:22.271054 17631 raft_consensus.cc:482] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.271999 17631 raft_consensus.cc:504] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.274533 17631 leader_election.cc:288] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:22.295879 17403 raft_consensus.cc:2229] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:22.296420 17403 raft_consensus.cc:2258] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.298312 17403 ts_tablet_manager.cc:1905] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.302831 17708 tablet_replica.cc:330] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:22.303910 17708 raft_consensus.cc:2229] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:22.304423 17708 raft_consensus.cc:2258] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.306154 17708 ts_tablet_manager.cc:1905] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.306398 17697 tablet.cc:1722] T c204e5fffe4f490295a12b8c93818c3c P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260923 17:31:22.309360 16827 consensus_peers.cc:487] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:22.320004 17694 tablet.cc:1722] T c204e5fffe4f490295a12b8c93818c3c P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:22.328946 17144 ts_tablet_manager.cc:1428] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.149s user 0.004s sys 0.002s
I20260923 17:31:22.330236 17144 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:22.332372 17697 tablet.cc:1722] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:22.337284 17725 tablet.cc:1722] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:22.340690 16718 tablet_service.cc:1467] Processing CreateTablet for tablet ef890c51e92244f4b97c490f24250679 (DEFAULT_TABLE table=default.Table_7eeb705b716645669b9ce77150103ab9 [id=0b18a95235434180ac58aae4df832c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.341728 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef890c51e92244f4b97c490f24250679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.345814 16864 tablet_service.cc:1467] Processing CreateTablet for tablet a71e9d28008c4670b3d48f32161d6aee (DEFAULT_TABLE table=default.Table_2b5ca9111a5a4876bf1ac5b3c3f857a9 [id=5378eca135cd4ae3a3a6377012d21a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.347000 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71e9d28008c4670b3d48f32161d6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.306538 17631 raft_consensus.cc:482] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.355175 17631 raft_consensus.cc:504] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.357065 17631 leader_election.cc:288] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:22.314534 17685 leader_election.cc:288] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:22.322851 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 78398.6us
I20260923 17:31:22.341434 17010 tablet_service.cc:1467] Processing CreateTablet for tablet ef890c51e92244f4b97c490f24250679 (DEFAULT_TABLE table=default.Table_7eeb705b716645669b9ce77150103ab9 [id=0b18a95235434180ac58aae4df832c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.341516 17013 tablet_service.cc:1467] Processing CreateTablet for tablet a71e9d28008c4670b3d48f32161d6aee (DEFAULT_TABLE table=default.Table_2b5ca9111a5a4876bf1ac5b3c3f857a9 [id=5378eca135cd4ae3a3a6377012d21a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.359573 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71e9d28008c4670b3d48f32161d6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.364876 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef890c51e92244f4b97c490f24250679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.387244 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.410348 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.412601 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 8a2f1143dcd64b2dbd25e17787096903 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.413648 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a2f1143dcd64b2dbd25e17787096903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.422183 17143 tablet_bootstrap.cc:654] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:22.444310 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260923 17:31:22.407285 16726 tablet_service.cc:1467] Processing CreateTablet for tablet a71e9d28008c4670b3d48f32161d6aee (DEFAULT_TABLE table=default.Table_2b5ca9111a5a4876bf1ac5b3c3f857a9 [id=5378eca135cd4ae3a3a6377012d21a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.408502 16723 tablet_service.cc:1467] Processing CreateTablet for tablet 8a2f1143dcd64b2dbd25e17787096903 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.425663 17144 tablet_bootstrap.cc:654] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:22.444006 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:22.450093 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet d2796fcb72f5497a92208b40224739e9 (table default.Table_632f010f9d114ecbab7901aa664556ac [id=17f994d2e127410685fffe47828a215b]). Expected version 1 got 0
I20260923 17:31:22.450752 16743 raft_consensus.cc:3037] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.450815 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.452584 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
W20260923 17:31:22.455744 16972 leader_election.cc:341] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.458022 16743 raft_consensus.cc:2454] T 191292e60d0d46fb89b52390feb48f80 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:22.383857 17709 raft_consensus.cc:504] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.465667 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71e9d28008c4670b3d48f32161d6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.454576 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:22.468742 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a2f1143dcd64b2dbd25e17787096903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.469311 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:22.474565 16827 leader_election.cc:341] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:22.468093 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
I20260923 17:31:22.477229 17144 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:22.477684 17144 ts_tablet_manager.cc:1397] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.148s user 0.012s sys 0.002s
I20260923 17:31:22.483932 17144 raft_consensus.cc:348] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.484668 17144 raft_consensus.cc:374] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.484885 17144 raft_consensus.cc:729] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.488541 16386 catalog_manager.cc:5495] T 3db3bfd8b67a4290a007de5b4b9d0e2e P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 1 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:22.489511 17144 consensus_queue.cc:260] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.492736 16718 tablet_service.cc:1467] Processing CreateTablet for tablet ba0ed72a55d74ab5b8b22fba90848094 (DEFAULT_TABLE table=default.Table_ca364ecd26d844f8b7394a4f3770b1b0 [id=a046b5cd73df44dbae49680318a82caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.493813 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba0ed72a55d74ab5b8b22fba90848094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.496102 17403 ts_tablet_manager.cc:1918] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260923 17:31:22.500037 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 44328us
I20260923 17:31:22.469741 17011 tablet_service.cc:1467] Processing CreateTablet for tablet ba0ed72a55d74ab5b8b22fba90848094 (DEFAULT_TABLE table=default.Table_ca364ecd26d844f8b7394a4f3770b1b0 [id=a046b5cd73df44dbae49680318a82caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.493642 17012 tablet_service.cc:1467] Processing CreateTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 (DEFAULT_TABLE table=default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.504016 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48bda995b284cf6be7bb6f348ec0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.464519 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 8a2f1143dcd64b2dbd25e17787096903 (DEFAULT_TABLE table=default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.509413 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a2f1143dcd64b2dbd25e17787096903. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:22.502638 16972 leader_election.cc:341] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.519595 16972 leader_election.cc:302] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:22.520355 17719 raft_consensus.cc:2781] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:22.516152 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba0ed72a55d74ab5b8b22fba90848094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.522356 17719 raft_consensus.cc:686] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:22.523216 17719 consensus_queue.cc:237] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.544960 17732 tablet.cc:1722] T c204e5fffe4f490295a12b8c93818c3c P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:22.513293 17144 ts_tablet_manager.cc:1428] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.035s user 0.007s sys 0.000s
I20260923 17:31:22.546422 17403 log.cc:1196] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/547fda6e7bcf474e9f73ba9a6659d34f
W20260923 17:31:22.549579 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 92741.6us
I20260923 17:31:22.555209 16869 tablet_service.cc:1467] Processing CreateTablet for tablet ba0ed72a55d74ab5b8b22fba90848094 (DEFAULT_TABLE table=default.Table_ca364ecd26d844f8b7394a4f3770b1b0 [id=a046b5cd73df44dbae49680318a82caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.556329 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba0ed72a55d74ab5b8b22fba90848094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.557518 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.561538 16870 tablet_service.cc:1467] Processing CreateTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 (DEFAULT_TABLE table=default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.562676 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48bda995b284cf6be7bb6f348ec0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.535105 16718 tablet_service.cc:1467] Processing CreateTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 (DEFAULT_TABLE table=default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.571349 17403 ts_tablet_manager.cc:1939] T 547fda6e7bcf474e9f73ba9a6659d34f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:22.571619 16718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48bda995b284cf6be7bb6f348ec0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:22.550341 16972 leader_election.cc:341] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.546651 17144 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:22.572072 16972 leader_election.cc:302] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:22.572734 17685 raft_consensus.cc:2726] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:22.573792 16972 leader_election.cc:341] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.534045 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 7c115e29673747f8b0712d441ac993fb (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.576915 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c115e29673747f8b0712d441ac993fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.578943 17542 ts_tablet_manager.cc:1918] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:22.579289 17542 log.cc:1196] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:22.579330 17733 tablet.cc:1722] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:22.579999 17542 ts_tablet_manager.cc:1939] T a80034cbf6354e1bbe0a0220e43251c3 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:22.552714 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a60bf118082b4e9991bfeb3ac59eb118" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:22.580596 16743 raft_consensus.cc:3037] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.590950 17146 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:22.591361 17146 ts_tablet_manager.cc:1397] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.496s user 0.006s sys 0.003s
I20260923 17:31:22.593644 17144 tablet_bootstrap.cc:654] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:22.598968 17146 raft_consensus.cc:348] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.599596 17146 raft_consensus.cc:374] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.599825 17146 raft_consensus.cc:729] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:22.602104 16972 leader_election.cc:341] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.603951 16972 leader_election.cc:302] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:22.606926 16743 raft_consensus.cc:2454] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:22.608156 16826 leader_election.cc:302] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:22.609652 17719 raft_consensus.cc:2726] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:22.612087 17146 consensus_queue.cc:260] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.614243 17697 tablet.cc:1717] T fba670ca9ed0470e95dc7eb77bca73e4 P be93fda0843b44f5abc49e6f35c69217: Skipping requested alter to schema version 2, tablet already version 2
I20260923 17:31:22.631765 17013 tablet_service.cc:1467] Processing CreateTablet for tablet 7c115e29673747f8b0712d441ac993fb (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.632273 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 147) took 2937 ms. Trace:
I20260923 17:31:22.632699 17013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c115e29673747f8b0712d441ac993fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.634552 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (DEFAULT_TABLE table=default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.635506 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.632575 17403 rpcz_store.cc:276] 0923 17:31:19.695126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.710334 (+ 15208us) service_pool.cc:227] Handling call
0923 17:31:22.228668 (+2518334us) ts_tablet_manager.cc:1145] Deleting tablet 547fda6e7bcf474e9f73ba9a6659d34f
0923 17:31:22.228727 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.323092 (+ 94365us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.495906 (+172814us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.632187 (+136281us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66229,"tablet-delete.queue_time_us":2485980}
I20260923 17:31:22.643108 17403 tablet_replica.cc:330] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:22.643270 17144 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:22.643731 17144 ts_tablet_manager.cc:1397] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.097s user 0.012s sys 0.000s
I20260923 17:31:22.649259 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (DEFAULT_TABLE table=default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.650321 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.652456 17144 raft_consensus.cc:348] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.653254 17719 raft_consensus.cc:482] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260923 17:31:22.653939 16678 consensus_peers.cc:487] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:22.653688 17719 raft_consensus.cc:504] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.655508 17719 leader_election.cc:288] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:22.657333 17144 raft_consensus.cc:374] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.658900 17144 raft_consensus.cc:729] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.664999 17144 consensus_queue.cc:260] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.552271 17709 leader_election.cc:288] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.671080 17144 ts_tablet_manager.cc:1428] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260923 17:31:22.674645 17144 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:22.677587 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.678201 16743 raft_consensus.cc:2454] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:22.679626 16972 leader_election.cc:302] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:22.681686 17709 raft_consensus.cc:2781] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:22.682229 17709 raft_consensus.cc:686] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:22.683059 17709 consensus_queue.cc:237] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.684857 17144 tablet_bootstrap.cc:654] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:22.640775 17664 raft_consensus.cc:482] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.690706 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:22.691491 17736 raft_consensus.cc:482] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.691634 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 85eba2aac8d84defa8ce4081d447bc63 (table default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95]). Expected version 1 got 0
I20260923 17:31:22.691732 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:22.692590 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:22.693120 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:22.695070 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:22.695865 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet c204e5fffe4f490295a12b8c93818c3c (table default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d]). Expected version 1 got 0
W20260923 17:31:22.698588 16972 leader_election.cc:341] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.690194 17664 raft_consensus.cc:504] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.700773 17664 leader_election.cc:288] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.705894 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet fba670ca9ed0470e95dc7eb77bca73e4 (table default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f]). Expected version 2 got 0
I20260923 17:31:22.708865 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d2796fcb72f5497a92208b40224739e9 (table default.Table_632f010f9d114ecbab7901aa664556ac [id=17f994d2e127410685fffe47828a215b]). Expected version 1 got 0
I20260923 17:31:22.714885 16384 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 186) took 2414 ms. Trace:
I20260923 17:31:22.691941 17736 raft_consensus.cc:504] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.725508 16717 tablet_service.cc:1467] Processing CreateTablet for tablet cfa712f9362e416e9cc6defcd8198bcd (DEFAULT_TABLE table=default.Table_9621239dc2864f8798ad1057b67f76c6 [id=7738bbb1031d4968837f61b539ce5b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.715221 16384 rpcz_store.cc:276] 0923 17:31:20.300823 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.301185 (+ 362us) service_pool.cc:227] Handling call
0923 17:31:20.308554 (+ 7369us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:20.309201 (+ 647us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:20.315247 (+ 6046us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:20.315573 (+ 326us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:20.370798 (+ 55225us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:22.714824 (+2344026us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:20.323870 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:20.337517 (+ 13647us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:20.337546 (+ 29us) write_op.cc:435] Acquired schema lock
0923 17:31:20.337558 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:20.337884 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:20.337922 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:20.337943 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:20.338097 (+ 154us) tablet.cc:699] Row locks acquired
0923 17:31:20.338109 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:20.338233 (+ 124us) write_op.cc:270] Start()
0923 17:31:20.338318 (+ 85us) write_op.cc:276] Timestamp: P: 1790184680338188 usec, L: 0
0923 17:31:20.338339 (+ 21us) op_driver.cc:331] REPLICATION: starting
0923 17:31:20.338879 (+ 540us) log.cc:842] Serialized 405 byte log entry
0923 17:31:20.355584 (+ 16705us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.357485 (+ 1901us) write_op.cc:301] APPLY: starting
0923 17:31:20.357532 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.357616 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.357728 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.357855 (+ 127us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.357970 (+ 115us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:20.358017 (+ 47us) write_op.cc:312] APPLY: finished
0923 17:31:20.358162 (+ 145us) log.cc:842] Serialized 23 byte log entry
0923 17:31:20.358264 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.358317 (+ 53us) write_op.cc:454] Released schema lock
0923 17:31:20.358349 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:20.358494 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.358516 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1799,"apply.run_cpu_time_us":1163,"apply.run_wall_time_us":1162,"num_ops":1,"prepare.queue_time_us":6869,"prepare.run_cpu_time_us":1964,"prepare.run_wall_time_us":16177,"replication_time_us":17189}]]}
I20260923 17:31:22.729493 17403 raft_consensus.cc:2229] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:22.730167 17403 raft_consensus.cc:2258] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.733304 16379 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f] alter complete (version 2)
I20260923 17:31:22.733963 17403 ts_tablet_manager.cc:1905] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:22.734515 16388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_89ce6a7e0e1c412aa40457102938869c [id=e8986668f81448488cbf75f8bf0e24f3] alter complete (version 1)
I20260923 17:31:22.736846 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 147) took 2680 ms. Trace:
I20260923 17:31:22.737306 17542 rpcz_store.cc:276] 0923 17:31:20.056661 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.056838 (+ 177us) service_pool.cc:227] Handling call
0923 17:31:22.032507 (+1975669us) ts_tablet_manager.cc:1145] Deleting tablet a80034cbf6354e1bbe0a0220e43251c3
0923 17:31:22.032551 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.353120 (+320569us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.578716 (+225596us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.736772 (+158056us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1975084}
I20260923 17:31:22.738781 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 148) took 2671 ms. Trace:
I20260923 17:31:22.739130 17542 rpcz_store.cc:276] 0923 17:31:20.067336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.067478 (+ 142us) service_pool.cc:227] Handling call
0923 17:31:22.738450 (+2670972us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:22.738488 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:22.738733 (+ 245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2665848}
I20260923 17:31:22.740644 17542 tablet_replica.cc:330] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:22.795044 17018 tablet_service.cc:1467] Processing CreateTablet for tablet cfa712f9362e416e9cc6defcd8198bcd (DEFAULT_TABLE table=default.Table_9621239dc2864f8798ad1057b67f76c6 [id=7738bbb1031d4968837f61b539ce5b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.796051 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa712f9362e416e9cc6defcd8198bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.799332 17146 ts_tablet_manager.cc:1428] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.208s user 0.004s sys 0.003s
I20260923 17:31:22.800192 17146 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:22.801081 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 27) took 2217 ms (client timeout 1000 ms). Trace:
W20260923 17:31:22.801682 16382 rpcz_store.cc:269] 0923 17:31:20.583998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.668319 (+ 84321us) service_pool.cc:227] Handling call
0923 17:31:22.800981 (+2132662us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":29}
W20260923 17:31:22.808080 16387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 26) took 3295 ms (client timeout 1000 ms). Trace:
W20260923 17:31:22.811512 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 30) took 3637 ms (client timeout 1000 ms). Trace:
I20260923 17:31:22.813731 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:22.817090 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.818256 17014 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.818746 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:22.820873 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 31) took 2630 ms (client timeout 1000 ms). Trace:
W20260923 17:31:22.821165 16388 rpcz_store.cc:269] 0923 17:31:20.190047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.190171 (+ 124us) service_pool.cc:227] Handling call
0923 17:31:20.197828 (+ 7657us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:22.820810 (+2622982us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:21.562848 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:21.563224 (+ 376us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:21.563247 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:21.563260 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:21.563521 (+ 261us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:21.563563 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:21.563584 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:21.563737 (+ 153us) tablet.cc:699] Row locks acquired
0923 17:31:21.563749 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:21.563860 (+ 111us) write_op.cc:270] Start()
0923 17:31:21.563944 (+ 84us) write_op.cc:276] Timestamp: P: 1790184681563804 usec, L: 0
0923 17:31:21.563960 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:21.583770 (+ 19810us) log.cc:842] Serialized 356 byte log entry
0923 17:31:21.624112 (+ 40342us) op_driver.cc:444] REPLICATION: finished
0923 17:31:21.624462 (+ 350us) write_op.cc:301] APPLY: starting
0923 17:31:21.624521 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:21.624613 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:21.624729 (+ 116us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:21.624890 (+ 161us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:21.625020 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:21.625064 (+ 44us) write_op.cc:312] APPLY: finished
0923 17:31:21.625256 (+ 192us) log.cc:842] Serialized 23 byte log entry
0923 17:31:21.625403 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.625469 (+ 66us) write_op.cc:454] Released schema lock
0923 17:31:21.625501 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:21.625674 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.625697 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23399,"reported_tablets":23,"spinlock_wait_cycles":9058560,"child_traces":[["op",{"apply.queue_time_us":195,"apply.run_cpu_time_us":1378,"apply.run_wall_time_us":1377,"num_ops":1,"prepare.queue_time_us":282,"prepare.run_cpu_time_us":2109,"prepare.run_wall_time_us":21296,"raft.queue_time_us":29689,"raft.run_cpu_time_us":707,"raft.run_wall_time_us":1177,"replication_time_us":60076}]]}
I20260923 17:31:22.830411 17009 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.833484 17016 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.834192 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.801213 17719 raft_consensus.cc:2781] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:22.841779 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.841756 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:22.754731 17144 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:22.847239 17144 ts_tablet_manager.cc:1397] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.175s user 0.011s sys 0.002s
I20260923 17:31:22.781991 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.791410 16715 tablet_service.cc:1514] Processing DeleteTablet for tablet d22f0f9d7e6a46a08ca9f411cc2c658b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.817513 16724 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.818356 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.819310 16713 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.837378 17719 raft_consensus.cc:482] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:22.849553 17144 raft_consensus.cc:348] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.851032 17719 raft_consensus.cc:3037] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.851146 17144 raft_consensus.cc:374] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:22.851704 17144 raft_consensus.cc:729] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:22.829015 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.837100 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa712f9362e416e9cc6defcd8198bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.861660 17144 consensus_queue.cc:260] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.865799 17144 ts_tablet_manager.cc:1428] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260923 17:31:22.867815 17144 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:22.887713 16677 consensus_peers.cc:487] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:22.746415 17736 leader_election.cc:288] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.773046 17725 tablet.cc:1717] T fba670ca9ed0470e95dc7eb77bca73e4 P c36e79694e154f84a46980719217d7f2: Skipping requested alter to schema version 2, tablet already version 2
W20260923 17:31:22.775079 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 166972us
I20260923 17:31:22.900566 16865 tablet_service.cc:1467] Processing CreateTablet for tablet cfa712f9362e416e9cc6defcd8198bcd (DEFAULT_TABLE table=default.Table_9621239dc2864f8798ad1057b67f76c6 [id=7738bbb1031d4968837f61b539ce5b2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.901561 16744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ccdf1bd7a64f51965e0a6dc7e033c9" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.901960 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ebfb3cfd3d49b39c7cf453199ac8bd" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:22.902107 16744 raft_consensus.cc:2454] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:22.902590 16745 raft_consensus.cc:2379] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:22.902683 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d65cfb2eb3648faab5869bba69cab3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.902938 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.903556 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.904008 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9b734b250d4920abbafe76f52ec540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.905019 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.907147 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.908840 17679 raft_consensus.cc:482] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:22.909225 17679 raft_consensus.cc:504] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.911293 17679 leader_election.cc:288] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:22.912899 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.913551 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 723000dc175844e990740dc503b53ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:11 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:22.913648 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:22.900566 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (DEFAULT_TABLE table=default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:22.912931 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:22.916097 16677 leader_election.cc:341] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.917018 17708 ts_tablet_manager.cc:1918] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:22.917438 17708 log.cc:1196] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:22.919243 17708 ts_tablet_manager.cc:1939] T f05a6f83314d433a9502e0c00ba8ad09 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:22.919651 17542 raft_consensus.cc:2229] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:22.919977 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa712f9362e416e9cc6defcd8198bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.920317 17035 raft_consensus.cc:2454] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:22.921231 17733 tablet.cc:1717] T fba670ca9ed0470e95dc7eb77bca73e4 P e5c071cdf65042a59cad226a15a5532c: Skipping requested alter to schema version 2, tablet already version 2
I20260923 17:31:22.922041 17542 raft_consensus.cc:2258] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:22.879930 17719 raft_consensus.cc:504] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.926338 16678 leader_election.cc:302] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:22.928570 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:22.929034 17719 leader_election.cc:288] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:22.930982 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:22.931069 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:22.931542 16745 raft_consensus.cc:3037] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.933451 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet f05a6f83314d433a9502e0c00ba8ad09 (table default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]) successfully deleted
W20260923 17:31:22.934139 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d because the tablet was not found. No further retry: Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
W20260923 17:31:22.935472 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 42ccd82f191942239ffb88459d362107 because the tablet was not found. No further retry: Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
I20260923 17:31:22.936264 17647 raft_consensus.cc:2781] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:22.936623 17647 raft_consensus.cc:482] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:22.936920 17647 raft_consensus.cc:3037] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:22.940181 16972 leader_election.cc:341] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:22.942998 16745 raft_consensus.cc:2454] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
W20260923 17:31:22.946674 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet e818f6192de6472cb27747c0e3a1f128 because the tablet was not found. No further retry: Not found: Tablet not found: e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:22.948941 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 7c115e29673747f8b0712d441ac993fb (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:22.949496 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a80034cbf6354e1bbe0a0220e43251c3 because the tablet was not found. No further retry: Not found: Tablet not found: a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:22.949872 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c115e29673747f8b0712d441ac993fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:22.952921 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because the tablet was not found. No further retry: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:22.953541 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a80034cbf6354e1bbe0a0220e43251c3 (table default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]) successfully deleted
W20260923 17:31:22.845438 16827 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
W20260923 17:31:22.954195 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 261d1d03290546969a74ca60beb890eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 261d1d03290546969a74ca60beb890eb already in progress: creating tablet
I20260923 17:31:22.955999 16826 leader_election.cc:302] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:22.956907 17736 raft_consensus.cc:2781] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:22.957233 17736 raft_consensus.cc:482] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:22.957508 17736 raft_consensus.cc:3037] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:22.963546 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 53500.4us
I20260923 17:31:22.964376 16826 leader_election.cc:302] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:22.965310 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_625e432250494eb799feeaad03f7112f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:22.967111 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_625e432250494eb799feeaad03f7112f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:22.808739 16387 rpcz_store.cc:269] 0923 17:31:19.512739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.636911 (+124172us) service_pool.cc:227] Handling call
0923 17:31:19.855213 (+218302us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:22.807996 (+2952783us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25448,"reported_tablets":28,"spinlock_wait_cycles":235178752}
I20260923 17:31:22.959378 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.961515 16713 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a182b6ced72428e9638465f0cc4e7ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:12 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.962237 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet a97e944f4e0c48db813e3bee61102c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:16 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.962693 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet a6f62eadc6a94aedb4a8164c545cda17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:22.973564 16972 leader_election.cc:302] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:22.974537 17719 raft_consensus.cc:2781] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:22.977387 17719 raft_consensus.cc:686] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:22.963095 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet ff22ffe24d424b1e8659579febfe1270 with delete_type TABLET_DATA_DELETED (replaced by tablet 051f2e478a324df98647d90107d55533) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:22.977792 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 051f2e478a324df98647d90107d55533 because the tablet was not found. No further retry: Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:22.946368 17647 raft_consensus.cc:504] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.978170 17719 consensus_queue.cc:237] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.979529 17737 raft_consensus.cc:3037] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.979976 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:22.980165 17647 leader_election.cc:288] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:22.982781 16677 leader_election.cc:341] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:22.813042 16380 rpcz_store.cc:269] 0923 17:31:19.173990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.280247 (+106257us) service_pool.cc:227] Handling call
0923 17:31:22.811452 (+3531205us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:20.242503 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:20.242833 (+ 330us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:20.242853 (+ 20us) write_op.cc:435] Acquired schema lock
0923 17:31:20.242865 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:20.243092 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:20.243125 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:20.243158 (+ 33us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:20.243280 (+ 122us) tablet.cc:699] Row locks acquired
0923 17:31:20.243292 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:20.243385 (+ 93us) write_op.cc:270] Start()
0923 17:31:20.243458 (+ 73us) write_op.cc:276] Timestamp: P: 1790184680243354 usec, L: 0
0923 17:31:20.243473 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:20.243896 (+ 423us) log.cc:842] Serialized 536 byte log entry
0923 17:31:20.266007 (+ 22111us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.268392 (+ 2385us) write_op.cc:301] APPLY: starting
0923 17:31:20.268436 (+ 44us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.268523 (+ 87us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.268607 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.268787 (+ 180us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.268917 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:20.268959 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:20.269121 (+ 162us) log.cc:842] Serialized 23 byte log entry
0923 17:31:20.269242 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.269295 (+ 53us) write_op.cc:454] Released schema lock
0923 17:31:20.269323 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:20.269532 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.269563 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76981,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":2258,"apply.run_cpu_time_us":1385,"apply.run_wall_time_us":1383,"num_ops":1,"prepare.queue_time_us":27567,"prepare.run_cpu_time_us":1608,"prepare.run_wall_time_us":1608,"raft.queue_time_us":1224,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":22459}]]}
I20260923 17:31:22.986225 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:22.986789 17039 raft_consensus.cc:3037] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:22.987394 17737 raft_consensus.cc:2726] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:22.989624 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 547fda6e7bcf474e9f73ba9a6659d34f because the tablet was not found. No further retry: Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
W20260923 17:31:22.993824 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet a6f62eadc6a94aedb4a8164c545cda17 because the tablet was not found. No further retry: Not found: Tablet not found: a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:22.993871 17039 raft_consensus.cc:2454] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
W20260923 17:31:22.996692 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 35) took 1598 ms (client timeout 1000 ms). Trace:
I20260923 17:31:22.967499 17736 raft_consensus.cc:504] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:22.893162 17143 tablet_bootstrap.cc:492] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:22.998710 16678 leader_election.cc:302] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
W20260923 17:31:22.997025 16385 rpcz_store.cc:269] 0923 17:31:21.398566 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:21.446396 (+ 47830us) service_pool.cc:227] Handling call
0923 17:31:22.441200 (+994804us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:22.441227 (+ 27us) write_op.cc:454] Released schema lock
0923 17:31:22.996623 (+555396us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:22.388097 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:22.388465 (+ 368us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:22.388486 (+ 21us) write_op.cc:435] Acquired schema lock
0923 17:31:22.388497 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:22.388729 (+ 232us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:22.388788 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:22.388809 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:22.388941 (+ 132us) tablet.cc:699] Row locks acquired
0923 17:31:22.388954 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:22.389064 (+ 110us) write_op.cc:270] Start()
0923 17:31:22.389144 (+ 80us) write_op.cc:276] Timestamp: P: 1790184682389031 usec, L: 0
0923 17:31:22.389160 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:22.389594 (+ 434us) log.cc:842] Serialized 536 byte log entry
0923 17:31:22.398234 (+ 8640us) op_driver.cc:444] REPLICATION: finished
0923 17:31:22.398548 (+ 314us) write_op.cc:301] APPLY: starting
0923 17:31:22.398601 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:22.398684 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:22.398778 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:22.398916 (+ 138us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:22.399060 (+ 144us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:22.399101 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:22.399255 (+ 154us) log.cc:842] Serialized 23 byte log entry
0923 17:31:22.399380 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:22.399437 (+ 57us) write_op.cc:454] Released schema lock
0923 17:31:22.399464 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":8340,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":183,"apply.run_cpu_time_us":1096,"apply.run_wall_time_us":1095,"num_ops":1,"prepare.queue_time_us":14878,"prepare.run_cpu_time_us":1748,"prepare.run_wall_time_us":1749,"raft.queue_time_us":1188,"raft.run_cpu_time_us":683,"raft.run_wall_time_us":683,"replication_time_us":9004}]]}
I20260923 17:31:22.999360 17143 ts_tablet_manager.cc:1397] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.930s user 0.004s sys 0.008s
I20260923 17:31:23.000023 17679 raft_consensus.cc:2781] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:23.001439 17679 raft_consensus.cc:686] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
W20260923 17:31:23.002029 16383 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 33) took 3824 ms (client timeout 1000 ms). Trace:
I20260923 17:31:23.002142 17679 consensus_queue.cc:237] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.002465 17143 raft_consensus.cc:348] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.003226 17143 raft_consensus.cc:374] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.003468 17143 raft_consensus.cc:729] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:22.954399 16827 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 108980us
I20260923 17:31:23.004127 17143 consensus_queue.cc:260] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.018579 17736 leader_election.cc:288] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:23.020884 16827 leader_election.cc:341] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:23.022557 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ccdf1bd7a64f51965e0a6dc7e033c9" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:23.023074 16745 raft_consensus.cc:3037] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.016816 17143 ts_tablet_manager.cc:1428] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260923 17:31:23.031832 17685 consensus_queue.cc:1035] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:23.040886 17685 consensus_queue.cc:1035] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:23.043666 17697 tablet.cc:1722] T d20c9b997a024151a810ea015c20e18f P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:23.057621 17745 tablet.cc:1722] T d20c9b997a024151a810ea015c20e18f P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:23.065502 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:23.066406 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:23.067905 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:23.074548 16388 master_service.cc:946] ReplaceTablet: received request to replace tablet a6f62eadc6a94aedb4a8164c545cda17 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:22.906095 17144 tablet_bootstrap.cc:654] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.077066 17143 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:23.084889 17753 raft_consensus.cc:482] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.085379 17753 raft_consensus.cc:504] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.088447 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:23.089084 17697 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260923 17:31:23.089651 16677 leader_election.cc:341] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.089948 17143 tablet_bootstrap.cc:654] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.092486 16866 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36510 (request call id 164) took 1042 ms. Trace:
I20260923 17:31:23.092868 16866 rpcz_store.cc:276] 0923 17:31:22.049883 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.050002 (+ 119us) service_pool.cc:227] Handling call
0923 17:31:22.051251 (+ 1249us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:22.051344 (+ 93us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:22.695403 (+644059us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:22.908506 (+213103us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:23.092424 (+183918us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260923 17:31:23.002528 16383 rpcz_store.cc:269] 0923 17:31:19.178156 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:19.320510 (+142354us) service_pool.cc:227] Handling call
0923 17:31:19.598952 (+278442us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:20.455064 (+856112us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.455096 (+ 32us) write_op.cc:454] Released schema lock
0923 17:31:23.001969 (+2546873us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:20.400951 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:20.401333 (+ 382us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:20.401357 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:20.401371 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:20.401644 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:20.401686 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:20.401708 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:20.401957 (+ 249us) tablet.cc:699] Row locks acquired
0923 17:31:20.401971 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:20.402076 (+ 105us) write_op.cc:270] Start()
0923 17:31:20.402157 (+ 81us) write_op.cc:276] Timestamp: P: 1790184680402033 usec, L: 0
0923 17:31:20.402173 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:20.402678 (+ 505us) log.cc:842] Serialized 536 byte log entry
0923 17:31:20.412445 (+ 9767us) op_driver.cc:444] REPLICATION: finished
0923 17:31:20.412846 (+ 401us) write_op.cc:301] APPLY: starting
0923 17:31:20.412907 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:20.413000 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:20.413097 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:20.413279 (+ 182us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:20.413415 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:20.413465 (+ 50us) write_op.cc:312] APPLY: finished
0923 17:31:20.413656 (+ 191us) log.cc:842] Serialized 23 byte log entry
0923 17:31:20.413803 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:20.413864 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:20.413900 (+ 36us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":150571,"reported_tablets":23,"spinlock_wait_cycles":250581248,"child_traces":[["op",{"apply.queue_time_us":259,"apply.run_cpu_time_us":1246,"apply.run_wall_time_us":1246,"num_ops":1,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":1959,"prepare.run_wall_time_us":44302,"raft.queue_time_us":91,"raft.run_cpu_time_us":745,"raft.run_wall_time_us":776,"replication_time_us":10196}]]}
I20260923 17:31:23.102116 17725 tablet.cc:1722] T d20c9b997a024151a810ea015c20e18f P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:23.103679 16383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ba9a40693fbc40cda7c43b3b9a5405f0" } modify_external_catalogs: true
I20260923 17:31:23.105866 17542 ts_tablet_manager.cc:1905] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.140785 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:23.149772 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:23.153575 16383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ba9a40693fbc40cda7c43b3b9a5405f0" } modify_external_catalogs: true
I20260923 17:31:23.154561 17143 tablet_bootstrap.cc:492] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
W20260923 17:31:23.131697 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 34) took 2913 ms (client timeout 1000 ms). Trace:
I20260923 17:31:23.155036 17143 ts_tablet_manager.cc:1397] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.078s user 0.007s sys 0.006s
W20260923 17:31:23.154879 16379 rpcz_store.cc:269] 0923 17:31:20.218210 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.218362 (+ 152us) service_pool.cc:227] Handling call
0923 17:31:20.232806 (+ 14444us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:21.529297 (+1296491us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.529327 (+ 30us) write_op.cc:454] Released schema lock
0923 17:31:23.131616 (+1602289us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:21.298743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:21.299116 (+ 373us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:21.299162 (+ 46us) write_op.cc:435] Acquired schema lock
0923 17:31:21.299175 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:21.299710 (+ 535us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:21.299752 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:21.299774 (+ 22us) tablet.cc:683] Acquiring locks for 5 operations
0923 17:31:21.300196 (+ 422us) tablet.cc:699] Row locks acquired
0923 17:31:21.300209 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:21.300346 (+ 137us) write_op.cc:270] Start()
0923 17:31:21.300427 (+ 81us) write_op.cc:276] Timestamp: P: 1790184681300306 usec, L: 0
0923 17:31:21.300440 (+ 13us) op_driver.cc:331] REPLICATION: starting
0923 17:31:21.300939 (+ 499us) log.cc:842] Serialized 2069 byte log entry
0923 17:31:21.311260 (+ 10321us) op_driver.cc:444] REPLICATION: finished
0923 17:31:21.311672 (+ 412us) write_op.cc:301] APPLY: starting
0923 17:31:21.311734 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:21.311895 (+ 161us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:21.312102 (+ 207us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:21.312604 (+ 502us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:21.312751 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0923 17:31:21.312798 (+ 47us) write_op.cc:312] APPLY: finished
0923 17:31:21.313018 (+ 220us) log.cc:842] Serialized 47 byte log entry
0923 17:31:21.313164 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.313247 (+ 83us) write_op.cc:454] Released schema lock
0923 17:31:21.313283 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:21.313464 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.313488 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:21.511877 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:21.512276 (+ 399us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:21.512309 (+ 33us) write_op.cc:435] Acquired schema lock
0923 17:31:21.512323 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:21.512591 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:21.512641 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:21.512665 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:21.512805 (+ 140us) tablet.cc:699] Row locks acquired
0923 17:31:21.512817 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:21.512940 (+ 123us) write_op.cc:270] Start()
0923 17:31:21.513024 (+ 84us) write_op.cc:276] Timestamp: P: 1790184681512886 usec, L: 0
0923 17:31:21.513039 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:21.513592 (+ 553us) log.cc:842] Serialized 356 byte log entry
0923 17:31:21.522131 (+ 8539us) op_driver.cc:444] REPLICATION: finished
0923 17:31:21.522442 (+ 311us) write_op.cc:301] APPLY: starting
0923 17:31:21.522502 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:21.522594 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:21.522708 (+ 114us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:21.522860 (+ 152us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:21.522989 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:21.523033 (+ 44us) write_op.cc:312] APPLY: finished
0923 17:31:21.523213 (+ 180us) log.cc:842] Serialized 23 byte log entry
0923 17:31:21.523359 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:21.523431 (+ 72us) write_op.cc:454] Released schema lock
0923 17:31:21.523461 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":326935,"reported_tablets":26,"spinlock_wait_cycles":23607680,"child_traces":[["op",{"apply.queue_time_us":265,"apply.run_cpu_time_us":2043,"apply.run_wall_time_us":2042,"num_ops":5,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":2495,"prepare.run_wall_time_us":2495,"raft.queue_time_us":2937,"raft.run_cpu_time_us":643,"raft.run_wall_time_us":643,"replication_time_us":10739}],["op",{"apply.queue_time_us":167,"apply.run_cpu_time_us":1190,"apply.run_wall_time_us":1189,"num_ops":1,"prepare.queue_time_us":23824,"prepare.run_cpu_time_us":2008,"prepare.run_wall_time_us":2010,"raft.queue_time_us":223,"raft.run_cpu_time_us":695,"raft.run_wall_time_us":694,"replication_time_us":9017}]]}
I20260923 17:31:23.163712 17759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:23.176375 17759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:23.181382 17753 leader_election.cc:288] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.183195 16745 raft_consensus.cc:2454] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:23.184991 16826 leader_election.cc:302] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:23.185231 17143 raft_consensus.cc:348] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.185878 17143 raft_consensus.cc:374] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.186112 17143 raft_consensus.cc:729] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.186640 17143 consensus_queue.cc:260] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.187160 17736 raft_consensus.cc:2781] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:23.187661 17736 raft_consensus.cc:686] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:23.188445 17736 consensus_queue.cc:237] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.193008 17725 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:23.202000 17732 tablet.cc:1722] T e2fa2e2d002746c8ba2b0dcf45372aa8 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260923 17:31:23.224424 16452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260923 17:31:23.237393 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:23.238212 17035 raft_consensus.cc:2454] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:23.240542 16452 leader_election.cc:334] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111)
I20260923 17:31:23.245329 16379 master_service.cc:946] ReplaceTablet: received request to replace tablet a6f62eadc6a94aedb4a8164c545cda17 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:23.246598 16379 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 200) took 1982 ms. Trace:
I20260923 17:31:23.259143 17146 tablet_bootstrap.cc:654] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.265769 17759 leader_election.cc:288] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 012948ec54c1441381b43989690264ce (127.15.246.188:46075)
I20260923 17:31:23.281525 17403 ts_tablet_manager.cc:1918] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:23.281965 17403 log.cc:1196] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:23.134884 16388 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 197) took 2023 ms. Trace:
I20260923 17:31:23.282349 17146 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
W20260923 17:31:23.162299 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 32) took 1616 ms (client timeout 1000 ms). Trace:
I20260923 17:31:23.179895 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70172383e8bf40a49974968e8a8f5b23" candidate_term: 4 candidate_status { last_received { term: 3 index: 232 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
W20260923 17:31:23.192476 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 51981.2us
I20260923 17:31:23.282866 17146 ts_tablet_manager.cc:1397] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.483s user 0.005s sys 0.009s
I20260923 17:31:23.284149 16454 leader_election.cc:302] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70172383e8bf40a49974968e8a8f5b23; no voters: 012948ec54c1441381b43989690264ce, 1dbd0d3324414f26ab5d34c375921ef2
I20260923 17:31:23.144634 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_acbd237bfa684410a9a16aec1821e400"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:23.285847 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acbd237bfa684410a9a16aec1821e400 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.285472 17146 raft_consensus.cc:348] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.286229 17146 raft_consensus.cc:374] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.286485 17146 raft_consensus.cc:729] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:23.285141 16386 rpcz_store.cc:269] 0923 17:31:21.546174 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:21.556863 (+ 10689us) service_pool.cc:227] Handling call
0923 17:31:21.618435 (+ 61572us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295ddc3e 00007f9127114fb4 00007f9127114f42 00007f9127114e71 00007f9127114dec 00007f91271140a2 00007f91253be94c 00007f91253bdd86
0923 17:31:23.162228 (+1543793us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:22.611670 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:22.612030 (+ 360us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:22.612056 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:22.612068 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:22.612355 (+ 287us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:22.612396 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:22.612417 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:22.612549 (+ 132us) tablet.cc:699] Row locks acquired
0923 17:31:22.612562 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:22.612668 (+ 106us) write_op.cc:270] Start()
0923 17:31:22.612748 (+ 80us) write_op.cc:276] Timestamp: P: 1790184682612617 usec, L: 0
0923 17:31:22.612764 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:22.613240 (+ 476us) log.cc:842] Serialized 536 byte log entry
0923 17:31:22.624281 (+ 11041us) op_driver.cc:444] REPLICATION: finished
0923 17:31:22.624630 (+ 349us) write_op.cc:301] APPLY: starting
0923 17:31:22.624685 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:22.624769 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:22.624875 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:22.625029 (+ 154us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:22.625148 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:22.625186 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:22.625347 (+ 161us) log.cc:842] Serialized 23 byte log entry
0923 17:31:22.625480 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:22.625539 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:22.625567 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:22.625724 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:22.625746 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4022,"reported_tablets":24,"spinlock_wait_cycles":79161088,"child_traces":[["op",{"apply.queue_time_us":223,"apply.run_cpu_time_us":1263,"apply.run_wall_time_us":1262,"mutex_wait_us":200,"num_ops":1,"prepare.queue_time_us":92863,"prepare.run_cpu_time_us":1953,"prepare.run_wall_time_us":2137,"raft.queue_time_us":485,"raft.run_cpu_time_us":690,"raft.run_wall_time_us":690,"replication_time_us":11427}]]}
I20260923 17:31:23.282398 16388 rpcz_store.cc:276] 0923 17:31:21.111584 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.074434 (+1962850us) service_pool.cc:227] Handling call
0923 17:31:23.134815 (+ 60381us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:23.282930 16379 rpcz_store.cc:276] 0923 17:31:21.264063 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.245214 (+1981151us) service_pool.cc:227] Handling call
0923 17:31:23.246538 (+ 1324us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:23.298532 17146 consensus_queue.cc:260] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.301702 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:23.303350 17146 ts_tablet_manager.cc:1428] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
W20260923 17:31:23.303289 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f05a6f83314d433a9502e0c00ba8ad09 because the tablet was not found. No further retry: Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:23.304252 17146 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.305083 17647 raft_consensus.cc:482] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.305491 17647 raft_consensus.cc:504] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.307209 17647 leader_election.cc:288] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.310878 17403 ts_tablet_manager.cc:1939] T a80034cbf6354e1bbe0a0220e43251c3 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:23.311602 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 150) took 3258 ms. Trace:
I20260923 17:31:23.311868 17403 rpcz_store.cc:276] 0923 17:31:20.053759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.053913 (+ 154us) service_pool.cc:227] Handling call
0923 17:31:22.642955 (+2589042us) ts_tablet_manager.cc:1145] Deleting tablet a80034cbf6354e1bbe0a0220e43251c3
0923 17:31:22.643008 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.118651 (+475643us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.281306 (+162655us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.311555 (+ 30249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2532408}
I20260923 17:31:23.312901 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a80034cbf6354e1bbe0a0220e43251c3 (table default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]) successfully deleted
W20260923 17:31:23.315877 16381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 28) took 1698 ms (client timeout 1000 ms). Trace:
I20260923 17:31:23.317641 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:23.319216 16677 leader_election.cc:341] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:23.320416 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 44713us
I20260923 17:31:23.320960 16678 leader_election.cc:302] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.321786 17144 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:23.322139 17144 ts_tablet_manager.cc:1397] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.454s user 0.009s sys 0.001s
I20260923 17:31:23.325119 17753 raft_consensus.cc:2781] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:23.325426 17753 raft_consensus.cc:482] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:23.325712 17753 raft_consensus.cc:3037] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:23.316550 16381 rpcz_store.cc:269] 0923 17:31:21.617661 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.163899 (+546238us) service_pool.cc:227] Handling call
0923 17:31:23.315816 (+1151917us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":29}
I20260923 17:31:23.332655 17146 tablet_bootstrap.cc:654] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.332928 17753 raft_consensus.cc:504] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.334740 17144 raft_consensus.cc:348] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.335350 17144 raft_consensus.cc:374] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.335609 17144 raft_consensus.cc:729] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.336452 17144 consensus_queue.cc:260] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.341054 17143 ts_tablet_manager.cc:1428] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.186s user 0.006s sys 0.000s
I20260923 17:31:23.341367 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:23.342643 16677 leader_election.cc:341] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.344069 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:23.344676 17035 raft_consensus.cc:2454] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:23.347224 17753 leader_election.cc:288] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.352809 17143 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:23.349468 16386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_841cfa422dfb457eb8f606f44dfa4c7c" } new_table_name: "default.Table_8be9935ea009465aaaae2c81f5fa6783" modify_external_catalogs: true
I20260923 17:31:23.352988 16379 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_005aa780a0d3449e928afd88676b3c66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:23.360414 16379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_005aa780a0d3449e928afd88676b3c66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.347898 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_421fb28f3b474d0899b8e4fc413b0a65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:23.364248 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_421fb28f3b474d0899b8e4fc413b0a65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.366047 17146 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:23.366588 17146 ts_tablet_manager.cc:1397] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.063s user 0.012s sys 0.000s
I20260923 17:31:23.367318 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3542e5d4b3e34da19c70ab50d8a95080" } modify_external_catalogs: true
I20260923 17:31:23.369211 17146 raft_consensus.cc:348] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.369962 17146 raft_consensus.cc:374] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.370193 17146 raft_consensus.cc:729] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.371399 17759 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70172383e8bf40a49974968e8a8f5b23 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260923 17:31:23.371042 17146 consensus_queue.cc:260] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.372650 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_3542e5d4b3e34da19c70ab50d8a95080" } modify_external_catalogs: true
I20260923 17:31:23.373780 17146 ts_tablet_manager.cc:1428] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260923 17:31:23.374593 17685 raft_consensus.cc:482] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.374758 17146 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.375063 17685 raft_consensus.cc:504] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.377238 17685 leader_election.cc:288] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:23.353972 17144 ts_tablet_manager.cc:1428] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260923 17:31:23.383648 17144 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:23.384541 17146 tablet_bootstrap.cc:654] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.379034 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:23.389271 16972 leader_election.cc:341] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.358888 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 151) took 3293 ms. Trace:
I20260923 17:31:23.380914 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:23.405525 16972 leader_election.cc:341] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.405877 17736 raft_consensus.cc:482] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.405926 16972 leader_election.cc:302] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.406261 17736 raft_consensus.cc:504] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.407126 17685 raft_consensus.cc:2726] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:23.408325 17403 rpcz_store.cc:276] 0923 17:31:20.065705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.065864 (+ 159us) service_pool.cc:227] Handling call
0923 17:31:23.358487 (+3292623us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:23.358549 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.358835 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3247701}
I20260923 17:31:23.409648 17736 leader_election.cc:288] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.410076 17146 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
W20260923 17:31:23.410082 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 261d1d03290546969a74ca60beb890eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 261d1d03290546969a74ca60beb890eb already in progress: creating tablet
I20260923 17:31:23.410581 17146 ts_tablet_manager.cc:1397] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.006s
I20260923 17:31:23.411330 17403 tablet_replica.cc:330] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
W20260923 17:31:23.413336 16826 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:23.413224 17146 raft_consensus.cc:348] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.413970 17146 raft_consensus.cc:374] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.414227 17146 raft_consensus.cc:729] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:23.414964 16827 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:23.415290 16827 leader_election.cc:302] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:23.415088 17146 consensus_queue.cc:260] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.417137 17403 raft_consensus.cc:2229] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:23.417423 17736 raft_consensus.cc:2726] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:23.419560 16373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260923 17:31:23.425161 17403 raft_consensus.cc:2258] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:23.426386 17685 raft_consensus.cc:482] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.427362 17685 raft_consensus.cc:504] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.428599 17403 ts_tablet_manager.cc:1905] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.429143 17685 leader_election.cc:288] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:23.431229 17146 ts_tablet_manager.cc:1428] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.020s user 0.007s sys 0.002s
I20260923 17:31:23.432412 17146 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.437158 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:23.438086 17146 tablet_bootstrap.cc:654] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.438153 16382 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 196) took 2383 ms. Trace:
W20260923 17:31:23.438402 16972 leader_election.cc:341] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.439992 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:23.440583 17035 raft_consensus.cc:3037] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.438452 16382 rpcz_store.cc:276] 0923 17:31:21.054733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.803666 (+1748933us) service_pool.cc:227] Handling call
0923 17:31:22.967845 (+164179us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:23.117692 (+149847us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:23.437386 (+319694us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:23.437838 (+ 452us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:23.438093 (+ 255us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.166740 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.167167 (+ 427us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.167194 (+ 27us) write_op.cc:435] Acquired schema lock
0923 17:31:23.167209 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:23.167512 (+ 303us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.167565 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.167590 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.167796 (+ 206us) tablet.cc:699] Row locks acquired
0923 17:31:23.167812 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:23.167941 (+ 129us) write_op.cc:270] Start()
0923 17:31:23.168024 (+ 83us) write_op.cc:276] Timestamp: P: 1790184683167870 usec, L: 0
0923 17:31:23.168043 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.168618 (+ 575us) log.cc:842] Serialized 433 byte log entry
0923 17:31:23.278338 (+109720us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.342321 (+ 63983us) write_op.cc:301] APPLY: starting
0923 17:31:23.364435 (+ 22114us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.364539 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.364660 (+ 121us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.365063 (+ 403us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.365217 (+ 154us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:23.365254 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:23.365424 (+ 170us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.437076 (+ 71652us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.437145 (+ 69us) write_op.cc:454] Released schema lock
0923 17:31:23.437178 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63775,"num_ops":2,"prepare.queue_time_us":47951,"prepare.run_cpu_time_us":2442,"prepare.run_wall_time_us":44330,"raft.queue_time_us":41575,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":56317,"replication_time_us":109970,"thread_start_us":156514,"threads_started":3}]]}
I20260923 17:31:23.452744 17685 consensus_queue.cc:1035] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:23.454857 17146 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:23.455302 17146 ts_tablet_manager.cc:1397] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.003s
I20260923 17:31:23.457722 17146 raft_consensus.cc:348] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.458490 17146 raft_consensus.cc:374] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.458734 17146 raft_consensus.cc:729] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.459335 17146 consensus_queue.cc:260] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.462399 17685 raft_consensus.cc:482] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.462837 17685 raft_consensus.cc:504] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.465893 17146 ts_tablet_manager.cc:1428] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260923 17:31:23.466823 17685 leader_election.cc:288] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:23.467542 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:23.468698 16972 leader_election.cc:341] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.469777 17736 raft_consensus.cc:482] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.470201 17736 raft_consensus.cc:504] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:23.474155 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 54610.4us
I20260923 17:31:23.439291 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:23.447693 16678 leader_election.cc:302] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.466356 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:23.476938 16972 leader_election.cc:341] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.477342 16972 leader_election.cc:302] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.477982 17685 raft_consensus.cc:2726] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:23.478338 16972 leader_election.cc:341] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.478688 16972 leader_election.cc:302] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.479297 17685 raft_consensus.cc:2726] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:23.466843 17146 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.480816 17603 raft_consensus.cc:2781] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:23.481129 17603 raft_consensus.cc:482] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:23.481398 17603 raft_consensus.cc:3037] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.485616 17146 tablet_bootstrap.cc:654] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:23.493366 16827 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
I20260923 17:31:23.503767 17144 tablet_bootstrap.cc:654] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.511355 17542 ts_tablet_manager.cc:1918] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:23.512212 17403 ts_tablet_manager.cc:1918] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:23.512604 17403 log.cc:1196] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:23.516465 17403 ts_tablet_manager.cc:1939] T e818f6192de6472cb27747c0e3a1f128 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:23.520604 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ccd82f191942239ffb88459d362107" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:23.521507 16745 raft_consensus.cc:2454] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:23.526346 17542 log.cc:1196] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:23.527235 17542 ts_tablet_manager.cc:1939] T e818f6192de6472cb27747c0e3a1f128 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:23.517714 17143 tablet_bootstrap.cc:700] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Time spent opening tablet: real 0.126s user 0.004s sys 0.000s
I20260923 17:31:23.528745 17143 tablet_bootstrap.cc:654] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.472018 17736 leader_election.cc:288] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.533159 16826 leader_election.cc:302] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:23.534327 17736 raft_consensus.cc:2781] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:23.534653 17736 raft_consensus.cc:482] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:23.535015 17736 raft_consensus.cc:3037] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.545867 17603 raft_consensus.cc:504] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.548792 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 149) took 2879 ms. Trace:
I20260923 17:31:23.551822 17144 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:23.553783 17144 ts_tablet_manager.cc:1397] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.170s user 0.009s sys 0.002s
I20260923 17:31:23.548955 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:23.559010 17542 rpcz_store.cc:276] 0923 17:31:20.669446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.669586 (+ 140us) service_pool.cc:227] Handling call
0923 17:31:22.740506 (+2070920us) ts_tablet_manager.cc:1145] Deleting tablet e818f6192de6472cb27747c0e3a1f128
0923 17:31:22.740545 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.243051 (+502506us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.511099 (+268048us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.548689 (+ 37590us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20369,"tablet-delete.queue_time_us":2070366}
W20260923 17:31:23.558531 16677 leader_election.cc:341] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.525767 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 153) took 2863 ms. Trace:
I20260923 17:31:23.563926 17403 rpcz_store.cc:276] 0923 17:31:20.662759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.662906 (+ 147us) service_pool.cc:227] Handling call
0923 17:31:23.411183 (+2748277us) ts_tablet_manager.cc:1145] Deleting tablet e818f6192de6472cb27747c0e3a1f128
0923 17:31:23.411231 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.491996 (+ 80765us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.512000 (+ 20004us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.525681 (+ 13681us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2747632}
I20260923 17:31:23.565188 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
I20260923 17:31:23.566169 17035 raft_consensus.cc:2454] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:23.566248 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c1fb4b15a00641eda2af13db76fd5ec6" } new_table_name: "default.Table_038736374c594b9e8129ba1c363b280a" modify_external_catalogs: true
I20260923 17:31:23.568395 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet e818f6192de6472cb27747c0e3a1f128 (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]) successfully deleted
I20260923 17:31:23.568917 17542 tablet_replica.cc:330] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:23.575901 16385 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 199) took 2337 ms. Trace:
I20260923 17:31:23.576247 16385 rpcz_store.cc:276] 0923 17:31:21.238446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.144150 (+1905704us) service_pool.cc:227] Handling call
0923 17:31:23.286241 (+142091us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:23.287576 (+ 1335us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:23.575307 (+287731us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:23.575604 (+ 297us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:23.575845 (+ 241us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.291169 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.291549 (+ 380us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.291573 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:23.291587 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:23.291873 (+ 286us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.291923 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.292044 (+ 121us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.335901 (+ 43857us) tablet.cc:699] Row locks acquired
0923 17:31:23.335917 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:23.336042 (+ 125us) write_op.cc:270] Start()
0923 17:31:23.336141 (+ 99us) write_op.cc:276] Timestamp: P: 1790184683336008 usec, L: 0
0923 17:31:23.336161 (+ 20us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.348068 (+ 11907us) log.cc:842] Serialized 433 byte log entry
0923 17:31:23.366607 (+ 18539us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:23.572900 (+206293us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.573279 (+ 379us) write_op.cc:301] APPLY: starting
0923 17:31:23.573333 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.573438 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.573457 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.573887 (+ 430us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.574037 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:23.574078 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:23.574255 (+ 177us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.574983 (+ 728us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.575051 (+ 68us) write_op.cc:454] Released schema lock
0923 17:31:23.575092 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.575282 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.575306 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":212,"apply.run_cpu_time_us":1630,"apply.run_wall_time_us":2199,"num_ops":2,"prepare.queue_time_us":2519,"prepare.run_cpu_time_us":2895,"prepare.run_wall_time_us":75553,"replication_time_us":236664,"spinlock_wait_cycles":26127232}]]}
I20260923 17:31:23.563014 17736 raft_consensus.cc:504] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.583060 17736 leader_election.cc:288] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.583480 17542 raft_consensus.cc:2229] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 17:31:23.583716 16373 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet a6894a4ee5fe4884b5162d8055e766fe consensus state has no leader
W20260923 17:31:23.584781 16827 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
I20260923 17:31:23.585632 17542 raft_consensus.cc:2258] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:23.590490 16678 leader_election.cc:302] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.592060 17403 tablet_replica.cc:330] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:23.592936 17403 raft_consensus.cc:2229] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:23.566560 17143 tablet_bootstrap.cc:492] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:23.593571 17143 ts_tablet_manager.cc:1397] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.241s user 0.010s sys 0.000s
I20260923 17:31:23.593974 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ccd82f191942239ffb88459d362107" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:23.592518 16794 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:23.593604 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:23.595762 17035 raft_consensus.cc:3037] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:15.028482 17354 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 17244
W20260923 17:31:23.599835 17354 kernel_stack_watchdog.cc:198] Thread 17244 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 278ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260923 17:31:13.742974 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:23.600975 17143 raft_consensus.cc:348] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.602165 17143 raft_consensus.cc:374] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.602443 17143 raft_consensus.cc:729] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:23.603498 16972 consensus_peers.cc:487] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:23.603682 17365 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 9854 milliseconds
I20260923 17:31:23.604036 17755 raft_consensus.cc:2781] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:23.604496 17755 raft_consensus.cc:686] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:23.589895 17542 ts_tablet_manager.cc:1905] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:23.605353 16826 leader_election.cc:341] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.605157 17755 consensus_queue.cc:237] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.605731 16826 leader_election.cc:302] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:23.606235 16826 raft_consensus.cc:2670] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:23.593030 17603 leader_election.cc:288] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:23.619459 17736 consensus_queue.cc:1035] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:23.630317 17035 raft_consensus.cc:2454] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
W20260923 17:31:23.631841 17244 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 9.883s user 0.009s sys 0.000s
W20260923 17:31:23.632287 17244 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 9.884s user 0.009s sys 0.000s
I20260923 17:31:23.635556 16678 leader_election.cc:302] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.636612 17603 raft_consensus.cc:2781] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
W20260923 17:31:23.636919 17369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:23.636993 17603 raft_consensus.cc:686] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:23.637454 17244 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260923 17:31:23.637694 17603 consensus_queue.cc:237] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.639249 17244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:23.637699 17403 raft_consensus.cc:2258] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:23.642763 17403 ts_tablet_manager.cc:1905] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.623844 17144 raft_consensus.cc:348] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.648586 17144 raft_consensus.cc:374] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.648845 17144 raft_consensus.cc:729] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.649483 17144 consensus_queue.cc:260] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.603192 17143 consensus_queue.cc:260] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.663321 17143 ts_tablet_manager.cc:1428] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.069s user 0.004s sys 0.000s
I20260923 17:31:23.665225 16379 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 201) took 2256 ms. Trace:
I20260923 17:31:23.665956 16388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 202) took 2017 ms. Trace:
W20260923 17:31:23.666596 16386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c929b8beaaf4c7aa668852b63af2204 consensus state has no leader
I20260923 17:31:23.667044 16386 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 203) took 1908 ms. Trace:
I20260923 17:31:23.658735 17144 ts_tablet_manager.cc:1428] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.096s user 0.004s sys 0.000s
I20260923 17:31:23.670742 17144 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:23.667277 16386 rpcz_store.cc:276] 0923 17:31:21.758760 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.290622 (+1531862us) service_pool.cc:227] Handling call
0923 17:31:23.349376 (+ 58754us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df7d6 00007f9127125634 00007f91271255c2 00007f91271254f1 00007f912712546c 00007f9127124722 00007f91253be94c 00007f91253bdd86
0923 17:31:23.367891 (+ 18515us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:23.373631 (+ 5740us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:23.413578 (+ 39947us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:23.415092 (+ 1514us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:23.666088 (+250996us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:23.666995 (+ 907us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.570846 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.571261 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.571286 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:23.571303 (+ 17us) tablet.cc:660] Decoding operations
0923 17:31:23.571593 (+ 290us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.571631 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.571655 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:23.571812 (+ 157us) tablet.cc:699] Row locks acquired
0923 17:31:23.571831 (+ 19us) write_op.cc:260] PREPARE: finished
0923 17:31:23.571943 (+ 112us) write_op.cc:270] Start()
0923 17:31:23.572028 (+ 85us) write_op.cc:276] Timestamp: P: 1790184683571897 usec, L: 0
0923 17:31:23.572047 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.572598 (+ 551us) log.cc:842] Serialized 405 byte log entry
0923 17:31:23.661372 (+ 88774us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.663137 (+ 1765us) write_op.cc:301] APPLY: starting
0923 17:31:23.663184 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.663260 (+ 76us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.663345 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.663494 (+ 149us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.663666 (+ 172us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:23.663712 (+ 46us) write_op.cc:312] APPLY: finished
0923 17:31:23.663905 (+ 193us) log.cc:842] Serialized 23 byte log entry
0923 17:31:23.664864 (+ 959us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.664918 (+ 54us) write_op.cc:454] Released schema lock
0923 17:31:23.664952 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.665143 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.665166 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":132078976,"child_traces":[["op",{"apply.queue_time_us":1657,"apply.run_cpu_time_us":1550,"apply.run_wall_time_us":2162,"num_ops":1,"prepare.queue_time_us":143415,"prepare.run_cpu_time_us":2039,"prepare.run_wall_time_us":4910,"raft.queue_time_us":2818,"raft.run_cpu_time_us":16,"raft.run_wall_time_us":16,"replication_time_us":89261,"spinlock_wait_cycles":1700736,"threads_started":1}]]}
I20260923 17:31:23.665537 16379 rpcz_store.cc:276] 0923 17:31:21.408434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.289343 (+1880909us) service_pool.cc:227] Handling call
0923 17:31:23.344875 (+ 55532us) spinlock_profiling.cc:243] Waited 54.7 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df156 00007f912711c034 00007f912711bfc2 00007f912711bef1 00007f912711be6c 00007f912711b122 00007f91253be94c 00007f91253bdd86
0923 17:31:23.360839 (+ 15964us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:23.362398 (+ 1559us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:23.664580 (+302182us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:23.664886 (+ 306us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:23.665164 (+ 278us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.563203 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.563660 (+ 457us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.563690 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:23.563701 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:23.564018 (+ 317us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.564064 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.564098 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.564345 (+ 247us) tablet.cc:699] Row locks acquired
0923 17:31:23.564358 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:23.568406 (+ 4048us) write_op.cc:270] Start()
0923 17:31:23.568537 (+ 131us) write_op.cc:276] Timestamp: P: 1790184683564433 usec, L: 0
0923 17:31:23.568551 (+ 14us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.569140 (+ 589us) log.cc:842] Serialized 433 byte log entry
0923 17:31:23.661188 (+ 92048us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.662310 (+ 1122us) write_op.cc:301] APPLY: starting
0923 17:31:23.662358 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.662452 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.662562 (+ 110us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.662933 (+ 371us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.663057 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:23.663113 (+ 56us) write_op.cc:312] APPLY: finished
0923 17:31:23.663258 (+ 145us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.664271 (+ 1013us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.664332 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:23.664360 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.664528 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.664546 (+ 18us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":125813120,"child_traces":[["op",{"apply.queue_time_us":995,"apply.run_cpu_time_us":2375,"apply.run_wall_time_us":2376,"num_ops":2,"prepare.queue_time_us":191024,"prepare.run_cpu_time_us":2364,"prepare.run_wall_time_us":7554,"raft.queue_time_us":1244,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":18,"replication_time_us":92563}]]}
I20260923 17:31:23.666301 16388 rpcz_store.cc:276] 0923 17:31:21.648249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.290227 (+1641978us) service_pool.cc:227] Handling call
0923 17:31:23.347739 (+ 57512us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df156 00007f912711c034 00007f912711bfc2 00007f912711bef1 00007f912711be6c 00007f912711b122 00007f91253be94c 00007f91253bdd86
0923 17:31:23.364663 (+ 16924us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:23.365938 (+ 1275us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:23.665347 (+299409us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:23.665648 (+ 301us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:23.665894 (+ 246us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.423344 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.423787 (+ 443us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.423813 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:23.423825 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:23.424120 (+ 295us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.424163 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.424188 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.472685 (+ 48497us) tablet.cc:699] Row locks acquired
0923 17:31:23.472702 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:23.472845 (+ 143us) write_op.cc:270] Start()
0923 17:31:23.472939 (+ 94us) write_op.cc:276] Timestamp: P: 1790184683472801 usec, L: 0
0923 17:31:23.472956 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.561393 (+ 88437us) log.cc:842] Serialized 433 byte log entry
0923 17:31:23.562807 (+ 1414us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:23.660957 (+ 98150us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.661386 (+ 429us) write_op.cc:301] APPLY: starting
0923 17:31:23.661449 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.661561 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.661592 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.662010 (+ 418us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.662144 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:23.662181 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:23.662381 (+ 200us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.662538 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.662621 (+ 83us) write_op.cc:454] Released schema lock
0923 17:31:23.662653 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.662838 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.662867 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":130684800,"child_traces":[["op",{"apply.queue_time_us":270,"apply.run_cpu_time_us":1639,"apply.run_wall_time_us":1638,"num_ops":2,"prepare.queue_time_us":53877,"prepare.run_cpu_time_us":4881,"prepare.run_wall_time_us":139686,"raft.queue_time_us":1781,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":17,"replication_time_us":187915,"spinlock_wait_cycles":201692800}]]}
I20260923 17:31:23.694085 17146 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:23.694085 16379 master_service.cc:946] ReplaceTablet: received request to replace tablet eb2beb8360d54efeb2fa74c005d011d1 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:23.694478 17146 ts_tablet_manager.cc:1397] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.228s user 0.048s sys 0.057s
I20260923 17:31:23.698599 17244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:23.702664 17143 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:23.714344 17146 raft_consensus.cc:348] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:23.719033 16827 consensus_peers.cc:487] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:23.717250 17146 raft_consensus.cc:374] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.720010 17146 raft_consensus.cc:729] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.725683 17146 consensus_queue.cc:260] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.727666 17143 tablet_bootstrap.cc:654] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.733762 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 (DEFAULT_TABLE table=default.Table_67ae78aa612d4ebcbaf1dcb60c98be71 [id=bbdc46a7e84c45198fc559139de7c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:23.734694 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd483bcad624c9a86503bdf15b9d6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:23.734722 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 (DEFAULT_TABLE table=default.Table_67ae78aa612d4ebcbaf1dcb60c98be71 [id=bbdc46a7e84c45198fc559139de7c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:23.735976 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd483bcad624c9a86503bdf15b9d6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:23.735714 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 (DEFAULT_TABLE table=default.Table_67ae78aa612d4ebcbaf1dcb60c98be71 [id=bbdc46a7e84c45198fc559139de7c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:23.736940 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_0655dcdf684c4131bc4d7118aca89c9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:23.738137 17146 ts_tablet_manager.cc:1428] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.043s user 0.006s sys 0.001s
W20260923 17:31:23.738481 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0655dcdf684c4131bc4d7118aca89c9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.738925 17146 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.740391 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd483bcad624c9a86503bdf15b9d6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:23.771311 17146 tablet_bootstrap.cc:654] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.773958 17403 ts_tablet_manager.cc:1918] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:23.774300 17403 log.cc:1196] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/42ccd82f191942239ffb88459d362107
W20260923 17:31:23.774708 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7dbd8eb0ab9243bd81c3d6a893f18aea consensus state has no leader
I20260923 17:31:23.774943 17403 ts_tablet_manager.cc:1939] T 42ccd82f191942239ffb88459d362107 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:23.775779 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 154) took 2965 ms. Trace:
I20260923 17:31:23.776105 17403 rpcz_store.cc:276] 0923 17:31:20.810423 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.810614 (+ 191us) service_pool.cc:227] Handling call
0923 17:31:23.584041 (+2773427us) ts_tablet_manager.cc:1145] Deleting tablet 42ccd82f191942239ffb88459d362107
0923 17:31:23.591948 (+ 7907us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.721744 (+129796us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.772774 (+ 51030us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.775729 (+ 2955us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2772766}
I20260923 17:31:23.777498 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 157) took 2840 ms. Trace:
I20260923 17:31:23.777781 17403 rpcz_store.cc:276] 0923 17:31:20.937229 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.937379 (+ 150us) service_pool.cc:227] Handling call
0923 17:31:23.777154 (+2839775us) ts_tablet_manager.cc:1145] Deleting tablet 006895667f324d049debbebb132ed09b
0923 17:31:23.777205 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.777456 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839330}
I20260923 17:31:23.778048 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 42ccd82f191942239ffb88459d362107 (table default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]) successfully deleted
I20260923 17:31:23.778653 17403 tablet_replica.cc:330] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
W20260923 17:31:23.779057 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 006895667f324d049debbebb132ed09b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 006895667f324d049debbebb132ed09b already in progress: creating tablet
I20260923 17:31:23.779299 17403 raft_consensus.cc:2229] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:23.781628 17143 tablet_bootstrap.cc:492] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:23.782136 17143 ts_tablet_manager.cc:1397] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.080s user 0.006s sys 0.004s
I20260923 17:31:23.785465 17143 raft_consensus.cc:348] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.785946 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_9595bead2a1e455aa4f250f726973485"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:23.786417 17143 raft_consensus.cc:374] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.786641 17143 raft_consensus.cc:729] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.787202 17143 consensus_queue.cc:260] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:23.787621 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9595bead2a1e455aa4f250f726973485 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.790321 17403 pending_rounds.cc:62] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:23.790606 17403 pending_rounds.cc:68] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596456469549056 op_type: NO_OP noop_request { }
I20260923 17:31:23.791240 17403 raft_consensus.cc:2866] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:23.791924 17403 raft_consensus.cc:2258] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:23.793948 17403 ts_tablet_manager.cc:1905] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.803783 17143 ts_tablet_manager.cc:1428] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.021s user 0.004s sys 0.002s
I20260923 17:31:23.810036 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
W20260923 17:31:23.810880 16677 consensus_peers.cc:487] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:23.816425 17685 raft_consensus.cc:482] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.816937 17685 raft_consensus.cc:504] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.818730 17685 leader_election.cc:288] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:23.821224 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:23.822878 16972 leader_election.cc:341] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.823812 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:23.825227 17736 raft_consensus.cc:482] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.825670 17736 raft_consensus.cc:504] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.827483 17736 leader_election.cc:288] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:23.828058 16972 leader_election.cc:341] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.828467 16972 leader_election.cc:302] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:23.829357 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:23.832384 17685 raft_consensus.cc:2726] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:23.836573 17143 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:23.824169 17403 ts_tablet_manager.cc:1918] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:23.843957 17403 log.cc:1196] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:23.830482 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191292e60d0d46fb89b52390feb48f80" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:23.825661 17144 tablet_bootstrap.cc:654] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.846388 17403 ts_tablet_manager.cc:1939] T f05a6f83314d433a9502e0c00ba8ad09 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:23.846390 16826 leader_election.cc:302] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:23.847651 17736 raft_consensus.cc:2726] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:23.847911 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 172) took 1684 ms. Trace:
I20260923 17:31:23.849135 17403 rpcz_store.cc:276] 0923 17:31:22.163682 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.170196 (+ 6514us) service_pool.cc:227] Handling call
0923 17:31:23.778528 (+1608332us) ts_tablet_manager.cc:1145] Deleting tablet f05a6f83314d433a9502e0c00ba8ad09
0923 17:31:23.778562 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.817616 (+ 39054us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.823984 (+ 6368us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.847838 (+ 23854us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1604910}
I20260923 17:31:23.850391 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 192) took 1059 ms. Trace:
I20260923 17:31:23.850296 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_dcf1d5122d0c421bab95f4f6e66d3f47 [id=6d073df311674a358a520dd1586a3345]). Expected version 3 got 2
I20260923 17:31:23.852849 17146 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:23.853238 17146 ts_tablet_manager.cc:1397] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.114s user 0.013s sys 0.015s
I20260923 17:31:23.855460 17146 raft_consensus.cc:348] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.856197 17146 raft_consensus.cc:374] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.856482 17146 raft_consensus.cc:729] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.857052 17403 rpcz_store.cc:276] 0923 17:31:22.790920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.791150 (+ 230us) service_pool.cc:227] Handling call
0923 17:31:23.850162 (+1059012us) ts_tablet_manager.cc:1145] Deleting tablet d22f0f9d7e6a46a08ca9f411cc2c658b
0923 17:31:23.850197 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.850349 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":999766}
I20260923 17:31:23.859301 17144 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:23.860111 17144 ts_tablet_manager.cc:1397] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.190s user 0.010s sys 0.005s
I20260923 17:31:23.862159 17144 raft_consensus.cc:348] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.862764 17144 raft_consensus.cc:374] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:23.863003 17144 raft_consensus.cc:729] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:23.863924 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 193) took 1046 ms. Trace:
I20260923 17:31:23.864244 17403 rpcz_store.cc:276] 0923 17:31:22.817133 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.817291 (+ 158us) service_pool.cc:227] Handling call
0923 17:31:23.863637 (+1046346us) ts_tablet_manager.cc:1145] Deleting tablet 389ecb50e0c34b4d92b5296bb17f4ab1
0923 17:31:23.863677 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.863853 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1013056}
I20260923 17:31:23.865320 17146 consensus_queue.cc:260] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.866514 17736 raft_consensus.cc:482] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.866938 17736 raft_consensus.cc:504] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.867347 17144 consensus_queue.cc:260] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.868775 17736 leader_election.cc:288] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:23.871281 16827 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:23.872385 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051f2e478a324df98647d90107d55533" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:23.872915 16745 raft_consensus.cc:2454] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 0.
I20260923 17:31:23.873983 16826 leader_election.cc:302] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters: e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:23.875413 17143 tablet_bootstrap.cc:654] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.875521 17736 raft_consensus.cc:2781] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:23.875844 17736 raft_consensus.cc:482] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:23.876130 17736 raft_consensus.cc:3037] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.877854 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 194) took 1059 ms. Trace:
I20260923 17:31:23.878139 17403 rpcz_store.cc:276] 0923 17:31:22.818068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.818222 (+ 154us) service_pool.cc:227] Handling call
0923 17:31:23.877597 (+1059375us) ts_tablet_manager.cc:1145] Deleting tablet 723000dc175844e990740dc503b53ced
0923 17:31:23.877638 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.877793 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1026884}
I20260923 17:31:23.850494 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet f05a6f83314d433a9502e0c00ba8ad09 (table default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]) successfully deleted
I20260923 17:31:23.883306 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_6974434dff19466482f1dccefd5689a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:23.884169 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet d22f0f9d7e6a46a08ca9f411cc2c658b because the tablet was not found. No further retry: Not found: Tablet not found: d22f0f9d7e6a46a08ca9f411cc2c658b
I20260923 17:31:23.864481 16382 master_service.cc:946] ReplaceTablet: received request to replace tablet 8e7c8e360d954ce78913e78c8662c341 from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:23.884778 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6974434dff19466482f1dccefd5689a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.886592 17144 ts_tablet_manager.cc:1428] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.026s user 0.004s sys 0.000s
I20260923 17:31:23.887229 17146 ts_tablet_manager.cc:1428] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.034s user 0.002s sys 0.004s
I20260923 17:31:23.888070 17146 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:23.888677 17403 tablet_replica.cc:330] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:23.889336 17403 raft_consensus.cc:2229] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:23.889753 17403 raft_consensus.cc:2258] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 17:31:23.884781 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 because the tablet was not found. No further retry: Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1
W20260923 17:31:23.890654 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 723000dc175844e990740dc503b53ced because the tablet was not found. No further retry: Not found: Tablet not found: 723000dc175844e990740dc503b53ced
I20260923 17:31:23.891893 17403 ts_tablet_manager.cc:1905] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.881181 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_4de36798f891433eb8cf23065562cfc2" } new_table_name: "default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95" modify_external_catalogs: true
I20260923 17:31:23.881359 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_f5da2cb10b2249359ef13479fa2c524d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:23.898898 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5da2cb10b2249359ef13479fa2c524d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:23.899964 17144 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:23.905105 17144 tablet_bootstrap.cc:654] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.919612 17403 ts_tablet_manager.cc:1918] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:23.910234 17736 raft_consensus.cc:504] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.923309 17403 log.cc:1196] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:23.924048 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051f2e478a324df98647d90107d55533" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:23.924336 17403 ts_tablet_manager.cc:1939] T 2c842eff86634bf4bc70c426b5226d66 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:23.924602 16745 raft_consensus.cc:3037] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.926898 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 189) took 1145 ms. Trace:
I20260923 17:31:23.927085 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:23.927160 17403 rpcz_store.cc:276] 0923 17:31:22.781372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.781584 (+ 212us) service_pool.cc:227] Handling call
0923 17:31:23.888537 (+1106953us) ts_tablet_manager.cc:1145] Deleting tablet 2c842eff86634bf4bc70c426b5226d66
0923 17:31:23.888577 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.908523 (+ 19946us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.919382 (+ 10859us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.926850 (+ 7468us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":364,"tablet-delete.queue_time_us":1028440}
I20260923 17:31:23.928308 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 195) took 1109 ms. Trace:
I20260923 17:31:23.928704 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 2c842eff86634bf4bc70c426b5226d66 (table default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]) successfully deleted
I20260923 17:31:23.928586 17403 rpcz_store.cc:276] 0923 17:31:22.818908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.819094 (+ 186us) service_pool.cc:227] Handling call
0923 17:31:23.928062 (+1108968us) ts_tablet_manager.cc:1145] Deleting tablet 3b9b734b250d4920abbafe76f52ec540
0923 17:31:23.928099 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.928260 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1077159}
W20260923 17:31:23.929672 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 3b9b734b250d4920abbafe76f52ec540 because the tablet was not found. No further retry: Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540
I20260923 17:31:23.934383 17403 tablet_replica.cc:330] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:23.935065 17403 raft_consensus.cc:2229] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:23.931061 17685 raft_consensus.cc:482] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:23.935469 17403 raft_consensus.cc:2258] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:23.936722 16745 raft_consensus.cc:2454] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:23.937866 16826 leader_election.cc:302] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:23.944453 17685 raft_consensus.cc:504] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:23.946930 17685 leader_election.cc:288] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:23.949724 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:23.950565 16739 raft_consensus.cc:2454] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:23.952932 16972 leader_election.cc:302] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:23.956799 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:23.960541 16972 leader_election.cc:341] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:23.966492 17244 hybrid_clock.cc:648] HybridClock initialized: now 1790184683966392 us; error 156 us; skew 500 ppm
I20260923 17:31:23.967805 17244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:23.973773 17685 raft_consensus.cc:2781] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:23.974272 17685 raft_consensus.cc:482] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:23.974609 17685 raft_consensus.cc:3037] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:23.983656 17146 tablet_bootstrap.cc:654] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:23.985478 17736 leader_election.cc:288] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:23.987653 16827 leader_election.cc:341] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:23.988358 17403 ts_tablet_manager.cc:1905] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:23.997506 17796 raft_consensus.cc:2781] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:23.997917 17796 raft_consensus.cc:686] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:23.998592 17796 consensus_queue.cc:237] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.010653 17403 ts_tablet_manager.cc:1918] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:24.011739 17403 log.cc:1196] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:24.012403 17403 ts_tablet_manager.cc:1939] T a60bf118082b4e9991bfeb3ac59eb118 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:24.016287 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 196) took 1187 ms. Trace:
I20260923 17:31:24.016590 17403 rpcz_store.cc:276] 0923 17:31:22.828676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.828857 (+ 181us) service_pool.cc:227] Handling call
0923 17:31:23.934255 (+1105398us) ts_tablet_manager.cc:1145] Deleting tablet a60bf118082b4e9991bfeb3ac59eb118
0923 17:31:23.934289 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.998993 (+ 64704us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.010436 (+ 11443us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.016239 (+ 5803us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12032,"tablet-delete.queue_time_us":1081953}
I20260923 17:31:24.017777 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a60bf118082b4e9991bfeb3ac59eb118 (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
I20260923 17:31:24.019690 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 198) took 1060 ms. Trace:
I20260923 17:31:24.020083 17403 rpcz_store.cc:276] 0923 17:31:22.959130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.959231 (+ 101us) service_pool.cc:227] Handling call
0923 17:31:24.019461 (+1060230us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:24.019512 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.019646 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1047355}
I20260923 17:31:24.020648 17737 consensus_queue.cc:1035] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:24.021689 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because the tablet was not found. No further retry: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:24.024490 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 200) took 1063 ms. Trace:
I20260923 17:31:24.024765 17403 rpcz_store.cc:276] 0923 17:31:22.961277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.961377 (+ 100us) service_pool.cc:227] Handling call
0923 17:31:24.023821 (+1062444us) ts_tablet_manager.cc:1145] Deleting tablet 1a182b6ced72428e9638465f0cc4e7ed
0923 17:31:24.023879 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.024443 (+ 564us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1051531}
W20260923 17:31:24.026588 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because the tablet was not found. No further retry: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:24.027683 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 201) took 1065 ms. Trace:
I20260923 17:31:24.027964 17403 rpcz_store.cc:276] 0923 17:31:22.962011 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.962100 (+ 89us) service_pool.cc:227] Handling call
0923 17:31:24.027479 (+1065379us) ts_tablet_manager.cc:1145] Deleting tablet a97e944f4e0c48db813e3bee61102c53
0923 17:31:24.027517 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.027640 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1055015}
W20260923 17:31:24.028959 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a97e944f4e0c48db813e3bee61102c53 because the tablet was not found. No further retry: Not found: Tablet not found: a97e944f4e0c48db813e3bee61102c53
I20260923 17:31:24.031037 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 202) took 1068 ms. Trace:
I20260923 17:31:24.031316 17403 rpcz_store.cc:276] 0923 17:31:22.962263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.962574 (+ 311us) service_pool.cc:227] Handling call
0923 17:31:24.030809 (+1068235us) ts_tablet_manager.cc:1145] Deleting tablet a6f62eadc6a94aedb4a8164c545cda17
0923 17:31:24.030852 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.030994 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39,"tablet-delete.queue_time_us":1058344}
W20260923 17:31:24.032356 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a6f62eadc6a94aedb4a8164c545cda17 because the tablet was not found. No further retry: Not found: Tablet not found: a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:24.034585 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 197) took 1075 ms. Trace:
I20260923 17:31:24.034901 17403 rpcz_store.cc:276] 0923 17:31:22.958886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.962993 (+ 4107us) service_pool.cc:227] Handling call
0923 17:31:24.034354 (+1071361us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:24.034391 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.034539 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1056225}
W20260923 17:31:24.038646 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because the tablet was not found. No further retry: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
W20260923 17:31:24.039098 16826 consensus_peers.cc:487] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:24.040704 17144 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:24.041812 17144 ts_tablet_manager.cc:1397] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.142s user 0.038s sys 0.020s
I20260923 17:31:24.044207 17144 raft_consensus.cc:348] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.044831 17144 raft_consensus.cc:374] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.044978 16382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 8e7c8e360d954ce78913e78c8662c341
I20260923 17:31:24.045047 17144 raft_consensus.cc:729] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.045796 17403 tablet_replica.cc:330] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:24.046308 16382 catalog_manager.cc:6565] ReplaceTablet: tablet 8e7c8e360d954ce78913e78c8662c341 deleted and replaced by tablet 8a992ad5280444dda64c1325da92a707
I20260923 17:31:24.046485 17403 raft_consensus.cc:2229] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:24.047222 17403 raft_consensus.cc:2258] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.047307 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_c925434b48d5441eab0ddbc29bed59f5" } new_table_name: "default.Table_bf7ef441cc5e439092b7d3133a346bc7" modify_external_catalogs: true
I20260923 17:31:24.049059 17403 ts_tablet_manager.cc:1905] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.049232 16386 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e0c75099d4d43339e88bf7527b10af9 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:24.050942 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:24.051083 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:24.053309 17144 consensus_queue.cc:260] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.055611 17144 ts_tablet_manager.cc:1428] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260923 17:31:24.056461 17144 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:24.061123 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a84e322a4de24cbabf450fceb098d6e7 consensus state has no leader
I20260923 17:31:24.061478 17144 tablet_bootstrap.cc:654] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.077214 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_fb7c50f5b9da47e98737dbf4f90005eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:24.080902 17755 consensus_queue.cc:1035] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:24.038635 17542 ts_tablet_manager.cc:1918] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:24.089489 17542 log.cc:1196] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/42ccd82f191942239ffb88459d362107
I20260923 17:31:24.091233 17542 ts_tablet_manager.cc:1939] T 42ccd82f191942239ffb88459d362107 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
W20260923 17:31:24.109319 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb7c50f5b9da47e98737dbf4f90005eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.114377 17144 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:24.115953 17144 ts_tablet_manager.cc:1397] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.060s user 0.009s sys 0.000s
I20260923 17:31:24.119575 17144 raft_consensus.cc:348] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.126091 17403 ts_tablet_manager.cc:1918] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:24.126441 17403 log.cc:1196] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:24.127075 17403 ts_tablet_manager.cc:1939] T eb2beb8360d54efeb2fa74c005d011d1 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:24.120178 17144 raft_consensus.cc:374] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.128535 17144 raft_consensus.cc:729] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.129757 17144 consensus_queue.cc:260] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.099947 17719 raft_consensus.cc:482] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.131139 17719 raft_consensus.cc:504] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:24.021807 16972 consensus_peers.cc:487] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:24.132121 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 110447us
W20260923 17:31:24.134097 16827 consensus_peers.cc:487] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:24.143429 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:24.143630 17719 leader_election.cc:288] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:24.144569 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_dcf1d5122d0c421bab95f4f6e66d3f47 [id=6d073df311674a358a520dd1586a3345]). Expected version 3 got 2
I20260923 17:31:24.144796 17719 raft_consensus.cc:482] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.145685 17719 raft_consensus.cc:504] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.148206 17719 leader_election.cc:288] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.153549 17144 ts_tablet_manager.cc:1428] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260923 17:31:24.156097 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.156008 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.158818 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.157452 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.053431 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:24.160400 17719 raft_consensus.cc:482] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.163265 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 204) took 1098 ms. Trace:
I20260923 17:31:24.164089 17719 raft_consensus.cc:504] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.165598 17403 rpcz_store.cc:276] 0923 17:31:23.065269 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.065364 (+ 95us) service_pool.cc:227] Handling call
0923 17:31:24.045657 (+980293us) ts_tablet_manager.cc:1145] Deleting tablet eb2beb8360d54efeb2fa74c005d011d1
0923 17:31:24.045698 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.069419 (+ 23721us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.125888 (+ 56469us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.163212 (+ 37324us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":979743}
I20260923 17:31:24.162066 17808 consensus_queue.cc:1035] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260923 17:31:24.165248 16972 leader_election.cc:341] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:24.168337 16972 leader_election.cc:341] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:24.168883 16972 leader_election.cc:341] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.169230 16972 leader_election.cc:302] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.170002 17808 raft_consensus.cc:2726] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.171011 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet eb2beb8360d54efeb2fa74c005d011d1 (table default.Table_49b13ac015354ceaa1f917b54abe16f6 [id=aed5416f9f034cb98f92051e19fad7a2]) successfully deleted
I20260923 17:31:24.163769 17144 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:24.111706 17146 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:24.174544 17146 ts_tablet_manager.cc:1397] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.287s user 0.014s sys 0.000s
I20260923 17:31:24.175343 17403 tablet_replica.cc:330] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:24.176013 17719 leader_election.cc:288] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.180783 17403 raft_consensus.cc:2229] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:24.099979 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 150) took 3288 ms. Trace:
W20260923 17:31:24.181569 16972 leader_election.cc:341] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.181998 16972 leader_election.cc:302] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.181512 17542 rpcz_store.cc:276] 0923 17:31:20.823113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.823255 (+ 142us) service_pool.cc:227] Handling call
0923 17:31:23.568750 (+2745495us) ts_tablet_manager.cc:1145] Deleting tablet 42ccd82f191942239ffb88459d362107
0923 17:31:23.568795 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:23.585309 (+ 16514us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b54000e1354. stack: 00007fe3d2037033 00007fe3d2036f30 00007fe3d1bb8534 00000000004db2b4 00000000004db249 00007fe3d8f43471 00007fe3d8fc6352 00007fe3d93dc26c 00007fe3d9fe9cfa 00007fe3da019e10 00007fe3da000e0d 00007fe3da000db9 00007fe3da000d49 00007fe3da000d11 00007fe3d9ffff3d 0000000000580784
0923 17:31:23.682311 (+ 97002us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:23.859157 (+176846us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.099908 (+240751us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13871488,"tablet-delete.queue_time_us":2744824}
I20260923 17:31:24.183917 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 42ccd82f191942239ffb88459d362107 (table default.Table_2402388e897844ae8d054926cdd0aa80 [id=6c35b720533640e79dfe4c8b420a9aab]) successfully deleted
I20260923 17:31:24.192555 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:24.194736 16972 leader_election.cc:341] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.195745 17719 raft_consensus.cc:2726] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.190757 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.197373 16745 raft_consensus.cc:2454] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:24.198549 16972 leader_election.cc:302] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.199218 17403 raft_consensus.cc:2258] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.200318 17719 raft_consensus.cc:2781] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.200904 17719 raft_consensus.cc:482] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.206836 17143 tablet_bootstrap.cc:492] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.015465 17685 raft_consensus.cc:504] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.207651 17719 raft_consensus.cc:3037] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.209128 17143 ts_tablet_manager.cc:1397] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.402s user 0.047s sys 0.051s
I20260923 17:31:24.211058 17685 leader_election.cc:288] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.212705 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:24.213289 16745 raft_consensus.cc:3037] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.213415 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 153) took 3276 ms. Trace:
I20260923 17:31:24.213743 17542 rpcz_store.cc:276] 0923 17:31:20.937517 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:20.937631 (+ 114us) service_pool.cc:227] Handling call
0923 17:31:24.183709 (+3246078us) ts_tablet_manager.cc:1145] Deleting tablet 006895667f324d049debbebb132ed09b
0923 17:31:24.183799 (+ 90us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.213350 (+ 29551us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3207076}
I20260923 17:31:24.214990 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 166) took 2052 ms. Trace:
W20260923 17:31:24.215157 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 006895667f324d049debbebb132ed09b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 006895667f324d049debbebb132ed09b already in progress: creating tablet
I20260923 17:31:24.215267 17542 rpcz_store.cc:276] 0923 17:31:22.162626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.162792 (+ 166us) service_pool.cc:227] Handling call
0923 17:31:24.214605 (+2051813us) ts_tablet_manager.cc:1145] Deleting tablet f05a6f83314d433a9502e0c00ba8ad09
0923 17:31:24.214654 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.214938 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1993362}
W20260923 17:31:24.216687 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f05a6f83314d433a9502e0c00ba8ad09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f05a6f83314d433a9502e0c00ba8ad09 already in progress: creating tablet
I20260923 17:31:24.216888 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:24.217675 17146 raft_consensus.cc:348] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.218809 17146 raft_consensus.cc:374] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.219062 17146 raft_consensus.cc:729] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.219679 17146 consensus_queue.cc:260] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.222257 16745 raft_consensus.cc:2454] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:24.224458 17403 ts_tablet_manager.cc:1905] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.226800 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 1e0c75099d4d43339e88bf7527b10af9
I20260923 17:31:24.228067 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet a86f4b7ee14948f0989af54b3b77dc71
I20260923 17:31:24.228567 16386 catalog_manager.cc:6565] ReplaceTablet: tablet 1e0c75099d4d43339e88bf7527b10af9 deleted and replaced by tablet b7b4d39a636e44e68ef2229006551ffa
I20260923 17:31:24.229157 16972 leader_election.cc:302] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:24.229353 16383 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 198) took 3088 ms. Trace:
I20260923 17:31:24.229854 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 with delete_type TABLET_DATA_DELETED (replaced by tablet b7b4d39a636e44e68ef2229006551ffa) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:24.229903 16972 leader_election.cc:341] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.230575 17685 raft_consensus.cc:2781] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.233454 17685 raft_consensus.cc:686] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:24.234534 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:24.234663 17808 raft_consensus.cc:482] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.235186 17808 raft_consensus.cc:504] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.237294 17808 leader_election.cc:288] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:24.229641 16383 rpcz_store.cc:276] 0923 17:31:21.141417 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.103385 (+1961968us) service_pool.cc:227] Handling call
0923 17:31:23.154078 (+ 50693us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:23.154588 (+ 510us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:23.154705 (+ 117us) catalog_manager.cc:2761] Locking tablets
0923 17:31:23.237278 (+ 82573us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:23.468958 (+231680us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:23.469191 (+ 233us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:24.227974 (+758783us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:24.229296 (+ 1322us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.239213 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.239594 (+ 381us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.239618 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:23.239633 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:23.239957 (+ 324us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.239996 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.240021 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.240301 (+ 280us) tablet.cc:699] Row locks acquired
0923 17:31:23.240316 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:23.240427 (+ 111us) write_op.cc:270] Start()
0923 17:31:23.240507 (+ 80us) write_op.cc:276] Timestamp: P: 1790184683240365 usec, L: 0
0923 17:31:23.240530 (+ 23us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.241010 (+ 480us) log.cc:842] Serialized 768 byte log entry
0923 17:31:23.404027 (+163017us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.457096 (+ 53069us) write_op.cc:301] APPLY: starting
0923 17:31:23.457160 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.457268 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.458231 (+ 963us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.458512 (+ 281us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.458667 (+ 155us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:23.458706 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:23.458909 (+ 203us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.462617 (+ 3708us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.462695 (+ 78us) write_op.cc:454] Released schema lock
0923 17:31:23.462732 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.469395 (+ 6663us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.469424 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":52931,"apply.run_cpu_time_us":2617,"apply.run_wall_time_us":12541,"num_ops":2,"prepare.queue_time_us":301,"prepare.run_cpu_time_us":2004,"prepare.run_wall_time_us":2003,"replication_time_us":163414,"thread_start_us":156161,"threads_started":1}]]}
I20260923 17:31:24.237751 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:24.229390 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 with delete_type TABLET_DATA_DELETED (replaced by tablet b7b4d39a636e44e68ef2229006551ffa) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:24.236963 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:24.239903 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_027bd79de4ee4fc7aee0f4c904532006"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:24.240391 17244 webserver.cc:460] Webserver started at http://127.15.246.188:33399/ using document root <none> and password file <none>
W20260923 17:31:24.241719 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_027bd79de4ee4fc7aee0f4c904532006 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.242004 17244 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:24.242347 17244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:24.242619 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 with delete_type TABLET_DATA_DELETED (replaced by tablet b7b4d39a636e44e68ef2229006551ffa) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:24.179927 17144 tablet_bootstrap.cc:654] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.244840 17143 raft_consensus.cc:348] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.247880 17685 consensus_queue.cc:237] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.245554 17143 raft_consensus.cc:374] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.249049 17143 raft_consensus.cc:729] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.249715 17143 consensus_queue.cc:260] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.259629 17808 raft_consensus.cc:482] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.260078 17808 raft_consensus.cc:504] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.261832 17808 leader_election.cc:288] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.263633 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.265637 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.267524 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.268270 16739 raft_consensus.cc:2454] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:24.269932 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:24.271402 16972 leader_election.cc:341] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.272032 16972 leader_election.cc:302] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:24.272822 16972 leader_election.cc:341] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.273290 16972 leader_election.cc:302] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.274187 17808 raft_consensus.cc:2781] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.274581 17808 raft_consensus.cc:482] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.274986 17808 raft_consensus.cc:3037] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.286695 17144 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:24.286963 17143 ts_tablet_manager.cc:1428] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.075s user 0.008s sys 0.000s
I20260923 17:31:24.287216 17144 ts_tablet_manager.cc:1397] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.126s user 0.009s sys 0.001s
I20260923 17:31:24.289415 17144 raft_consensus.cc:348] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.291772 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_022c670c0e8f41a4ae4d26c8a376a76b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:24.292564 17685 raft_consensus.cc:2726] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:24.294082 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_022c670c0e8f41a4ae4d26c8a376a76b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.296964 17719 raft_consensus.cc:504] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.299577 17719 leader_election.cc:288] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.304842 17144 raft_consensus.cc:374] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.305137 17144 raft_consensus.cc:729] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.306454 17143 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:24.310313 16972 leader_election.cc:341] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.311965 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:24.313411 16739 raft_consensus.cc:3037] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.314622 17810 tablet_replica.cc:330] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:24.317334 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 177) took 1414 ms. Trace:
I20260923 17:31:24.321378 16739 raft_consensus.cc:2454] T 151184083ebb4ae4bf24fe171d33cfab P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:24.329291 17144 consensus_queue.cc:260] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.341231 17244 fs_manager.cc:709] Time spent opening directory manager: real 0.032s user 0.004s sys 0.003s
I20260923 17:31:24.344592 17144 ts_tablet_manager.cc:1428] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.057s user 0.005s sys 0.000s
I20260923 17:31:24.327960 16972 leader_election.cc:302] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:24.347121 17815 raft_consensus.cc:2781] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.340504 17808 raft_consensus.cc:504] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.347560 17815 raft_consensus.cc:686] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:24.348311 17815 consensus_queue.cc:237] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.349072 17810 raft_consensus.cc:2229] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:24.349800 17542 rpcz_store.cc:276] 0923 17:31:22.902443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.902548 (+ 105us) service_pool.cc:227] Handling call
0923 17:31:24.216426 (+1313878us) ts_tablet_manager.cc:1145] Deleting tablet 8d65cfb2eb3648faab5869bba69cab3f
0923 17:31:24.216496 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.317262 (+100766us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1313258}
I20260923 17:31:24.323494 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:24.338141 17143 tablet_bootstrap.cc:654] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:24.351507 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 8d65cfb2eb3648faab5869bba69cab3f because the tablet was not found. No further retry: Not found: Tablet not found: 8d65cfb2eb3648faab5869bba69cab3f
I20260923 17:31:24.357364 17146 ts_tablet_manager.cc:1428] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.182s user 0.006s sys 0.000s
I20260923 17:31:24.359019 17146 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:24.362246 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 176) took 1460 ms. Trace:
I20260923 17:31:24.362608 17542 rpcz_store.cc:276] 0923 17:31:22.902137 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.902811 (+ 674us) service_pool.cc:227] Handling call
0923 17:31:24.361893 (+1459082us) ts_tablet_manager.cc:1145] Deleting tablet 1a182b6ced72428e9638465f0cc4e7ed
0923 17:31:24.361954 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.362170 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1458533}
W20260923 17:31:24.363988 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1a182b6ced72428e9638465f0cc4e7ed because the tablet was not found. No further retry: Not found: Tablet not found: 1a182b6ced72428e9638465f0cc4e7ed
I20260923 17:31:24.365692 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 175) took 1465 ms. Trace:
I20260923 17:31:24.366811 17808 leader_election.cc:288] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:24.368482 17143 tablet_bootstrap.cc:492] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.368918 17143 ts_tablet_manager.cc:1397] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.063s user 0.018s sys 0.001s
I20260923 17:31:24.371598 17143 raft_consensus.cc:348] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.372303 17143 raft_consensus.cc:374] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.372550 17143 raft_consensus.cc:729] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.373181 17143 consensus_queue.cc:260] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.375435 17143 ts_tablet_manager.cc:1428] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260923 17:31:24.376820 17821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:24.380295 17244 fs_manager.cc:725] Time spent opening block manager: real 0.036s user 0.008s sys 0.004s
I20260923 17:31:24.380753 17244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
uuid: "012948ec54c1441381b43989690264ce"
format_stamp: "Formatted at 2026-09-23 17:30:59 on dist-test-slave-mk2q"
I20260923 17:31:24.384976 17244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:24.390403 17143 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:24.399392 17810 raft_consensus.cc:2258] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.400349 17143 tablet_bootstrap.cc:654] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.400674 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:24.402184 17810 ts_tablet_manager.cc:1905] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.416536 17143 tablet_bootstrap.cc:492] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.417124 17143 ts_tablet_manager.cc:1397] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.005s
I20260923 17:31:24.419404 17143 raft_consensus.cc:348] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.423059 17143 raft_consensus.cc:374] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.423349 17143 raft_consensus.cc:729] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.423964 17143 consensus_queue.cc:260] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.426203 17143 ts_tablet_manager.cc:1428] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260923 17:31:24.427071 17143 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:24.430075 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_566b62c66bac46eb97bbae9720393eec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:24.385140 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53fd73fce64b658b09a0fbfcdbb0fd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:24.432924 16388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_7e13d4e4841d47c18ecc2442c83ec934" } new_table_name: "default.Table_61c9bac7f4414016b97e990deaa211c5" modify_external_catalogs: true
I20260923 17:31:24.433209 16745 raft_consensus.cc:3037] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.419589 17755 consensus_queue.cc:1035] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:24.424754 17603 raft_consensus.cc:482] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.437237 17603 raft_consensus.cc:504] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.438987 17603 leader_election.cc:288] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.441169 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:24.441730 17040 raft_consensus.cc:2454] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:24.444845 17797 raft_consensus.cc:482] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.445277 17797 raft_consensus.cc:504] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.447043 17797 leader_election.cc:288] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.449592 17143 tablet_bootstrap.cc:654] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.451112 16678 leader_election.cc:302] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:24.452093 16827 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:24.452790 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:24.454037 16677 leader_election.cc:341] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.454849 17603 raft_consensus.cc:2781] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.455219 17603 raft_consensus.cc:482] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.455540 17603 raft_consensus.cc:3037] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.366298 17542 rpcz_store.cc:276] 0923 17:31:22.900175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.903450 (+ 3275us) service_pool.cc:227] Handling call
0923 17:31:24.365185 (+1461735us) ts_tablet_manager.cc:1145] Deleting tablet 389ecb50e0c34b4d92b5296bb17f4ab1
0923 17:31:24.365267 (+ 82us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.365599 (+ 332us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1461215}
W20260923 17:31:24.464947 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 389ecb50e0c34b4d92b5296bb17f4ab1 because the tablet was not found. No further retry: Not found: Tablet not found: 389ecb50e0c34b4d92b5296bb17f4ab1
I20260923 17:31:24.466168 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 179) took 1562 ms. Trace:
I20260923 17:31:24.466642 17542 rpcz_store.cc:276] 0923 17:31:22.903782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.903870 (+ 88us) service_pool.cc:227] Handling call
0923 17:31:24.465870 (+1562000us) ts_tablet_manager.cc:1145] Deleting tablet 3b9b734b250d4920abbafe76f52ec540
0923 17:31:24.465919 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.466106 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1561444}
W20260923 17:31:24.469254 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 3b9b734b250d4920abbafe76f52ec540 because the tablet was not found. No further retry: Not found: Tablet not found: 3b9b734b250d4920abbafe76f52ec540
W20260923 17:31:24.475328 16677 consensus_peers.cc:487] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:24.476403 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_566b62c66bac46eb97bbae9720393eec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.477943 17679 raft_consensus.cc:482] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.478456 17679 raft_consensus.cc:504] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.480654 17679 leader_election.cc:288] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:24.354344 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 8e7c8e360d954ce78913e78c8662c341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e7c8e360d954ce78913e78c8662c341 already in progress: creating tablet
W20260923 17:31:24.481582 16373 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 127277us
I20260923 17:31:24.486662 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:24.487278 17035 raft_consensus.cc:2454] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:24.488281 16678 leader_election.cc:302] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:24.488988 17679 raft_consensus.cc:2781] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.489295 17679 raft_consensus.cc:482] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.489573 17679 raft_consensus.cc:3037] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.463994 16745 raft_consensus.cc:2454] T bd53fd73fce64b658b09a0fbfcdbb0fd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
W20260923 17:31:24.497701 16972 leader_election.cc:341] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.522720 17146 tablet_bootstrap.cc:654] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:24.523041 16972 leader_election.cc:341] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.473573 17143 tablet_bootstrap.cc:492] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.524765 17143 ts_tablet_manager.cc:1397] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.098s user 0.009s sys 0.001s
I20260923 17:31:24.524641 16972 leader_election.cc:302] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.525430 17808 raft_consensus.cc:2781] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.527053 17143 raft_consensus.cc:348] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.527707 17143 raft_consensus.cc:374] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.525905 17808 raft_consensus.cc:686] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:24.527951 17143 raft_consensus.cc:729] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:24.501444 16826 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:24.528645 16826 leader_election.cc:302] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:24.528530 17143 consensus_queue.cc:260] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.529479 17800 raft_consensus.cc:2726] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.529105 17808 consensus_queue.cc:237] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.529963 17143 ts_tablet_manager.cc:1428] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:24.482059 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:24.532915 16677 leader_election.cc:341] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.471789 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 180) took 1567 ms. Trace:
I20260923 17:31:24.535583 17542 rpcz_store.cc:276] 0923 17:31:22.904828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.904916 (+ 88us) service_pool.cc:227] Handling call
0923 17:31:24.471163 (+1566247us) ts_tablet_manager.cc:1145] Deleting tablet ff22ffe24d424b1e8659579febfe1270
0923 17:31:24.471223 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.471639 (+ 416us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1565797}
W20260923 17:31:24.536890 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet ff22ffe24d424b1e8659579febfe1270 because the tablet was not found. No further retry: Not found: Tablet not found: ff22ffe24d424b1e8659579febfe1270
I20260923 17:31:24.547886 17755 raft_consensus.cc:482] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.548404 17755 raft_consensus.cc:504] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.549546 17144 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:24.551342 17755 leader_election.cc:288] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.553115 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:24.553730 17035 raft_consensus.cc:2454] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:24.554713 16678 leader_election.cc:302] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:24.556500 17755 raft_consensus.cc:2781] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.556795 17755 raft_consensus.cc:482] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.557231 17755 raft_consensus.cc:3037] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.559821 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:24.562382 16677 leader_election.cc:341] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.563731 16383 master_service.cc:946] ReplaceTablet: received request to replace tablet c8895cffac5945a19a5de7fba1319f22 from {username='slave'} at 127.0.0.1:48560
W20260923 17:31:24.569383 16827 consensus_peers.cc:487] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:24.571229 17403 ts_tablet_manager.cc:1918] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:24.571604 17403 log.cc:1196] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:24.572304 17403 ts_tablet_manager.cc:1939] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:24.573963 17603 raft_consensus.cc:504] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.587391 17810 ts_tablet_manager.cc:1918] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:24.589015 17143 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:24.593590 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 181) took 1687 ms. Trace:
I20260923 17:31:24.593925 17542 rpcz_store.cc:276] 0923 17:31:22.906312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.907039 (+ 727us) service_pool.cc:227] Handling call
0923 17:31:24.536423 (+1629384us) ts_tablet_manager.cc:1145] Deleting tablet a6f62eadc6a94aedb4a8164c545cda17
0923 17:31:24.593281 (+ 56858us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.593518 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1628947}
W20260923 17:31:24.595368 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a6f62eadc6a94aedb4a8164c545cda17 because the tablet was not found. No further retry: Not found: Tablet not found: a6f62eadc6a94aedb4a8164c545cda17
I20260923 17:31:24.596494 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 182) took 1689 ms. Trace:
I20260923 17:31:24.596840 17542 rpcz_store.cc:276] 0923 17:31:22.906587 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.912765 (+ 6178us) service_pool.cc:227] Handling call
0923 17:31:24.596182 (+1683417us) ts_tablet_manager.cc:1145] Deleting tablet eb2beb8360d54efeb2fa74c005d011d1
0923 17:31:24.596254 (+ 72us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.596435 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1682884}
W20260923 17:31:24.598183 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because the tablet was not found. No further retry: Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:24.599252 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 178) took 1695 ms. Trace:
I20260923 17:31:24.599610 17542 rpcz_store.cc:276] 0923 17:31:22.903502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.913414 (+ 9912us) service_pool.cc:227] Handling call
0923 17:31:24.598986 (+1685572us) ts_tablet_manager.cc:1145] Deleting tablet 723000dc175844e990740dc503b53ced
0923 17:31:24.599026 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.599195 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1685042}
W20260923 17:31:24.600775 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 723000dc175844e990740dc503b53ced because the tablet was not found. No further retry: Not found: Tablet not found: 723000dc175844e990740dc503b53ced
I20260923 17:31:24.601667 17542 tablet_replica.cc:330] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:24.602506 17542 raft_consensus.cc:2229] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:24.580873 17753 consensus_queue.cc:1035] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:24.603577 17542 raft_consensus.cc:2258] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.605497 17542 ts_tablet_manager.cc:1905] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:24.607010 16677 consensus_peers.cc:487] T 373ad39e4d134ffc80caa73a83b80795 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:24.609470 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:24.610363 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 354f66c852b94b1d9f44b6bc012cf81b (DEFAULT_TABLE table=default.Table_421fb28f3b474d0899b8e4fc413b0a65 [id=b6361ecc549d4f53a624d46842a9444e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.610785 16886 raft_consensus.cc:3037] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.611275 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 354f66c852b94b1d9f44b6bc012cf81b (DEFAULT_TABLE table=default.Table_421fb28f3b474d0899b8e4fc413b0a65 [id=b6361ecc549d4f53a624d46842a9444e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.612828 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 354f66c852b94b1d9f44b6bc012cf81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.619828 16886 raft_consensus.cc:2454] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:24.620937 16677 leader_election.cc:302] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2; no voters:
I20260923 17:31:24.621670 17753 raft_consensus.cc:2781] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.622071 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 6e20fc94eecf4a14840ca88177070e33 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.622699 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 6e20fc94eecf4a14840ca88177070e33 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.623117 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e20fc94eecf4a14840ca88177070e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.622074 17753 raft_consensus.cc:686] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:24.626385 17603 leader_election.cc:288] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.626796 17143 tablet_bootstrap.cc:654] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.625159 17753 consensus_queue.cc:237] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.630262 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 354f66c852b94b1d9f44b6bc012cf81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.632632 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e20fc94eecf4a14840ca88177070e33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:24.636286 16388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2e5bc4ae186404c896814b95d6cc7ef consensus state has no leader
I20260923 17:31:24.645413 16388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_9595bead2a1e455aa4f250f726973485" } modify_external_catalogs: true
I20260923 17:31:24.646035 16388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_9595bead2a1e455aa4f250f726973485" } modify_external_catalogs: true
I20260923 17:31:24.656314 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 80b11a6fd84840178b81d2b7610c7351 (DEFAULT_TABLE table=default.Table_acbd237bfa684410a9a16aec1821e400 [id=84e44915fea341c2b33012883a1b3ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.657397 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b11a6fd84840178b81d2b7610c7351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.658419 17810 log.cc:1196] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/a86f4b7ee14948f0989af54b3b77dc71
I20260923 17:31:24.659286 17810 ts_tablet_manager.cc:1939] T a86f4b7ee14948f0989af54b3b77dc71 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:24.664281 17146 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:24.664815 17146 ts_tablet_manager.cc:1397] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.306s user 0.013s sys 0.002s
I20260923 17:31:24.670845 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.672009 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feebaa76bcd4ff5b707814191cb3341. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:24.592518 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 68001us
I20260923 17:31:24.615895 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 354f66c852b94b1d9f44b6bc012cf81b (DEFAULT_TABLE table=default.Table_421fb28f3b474d0899b8e4fc413b0a65 [id=b6361ecc549d4f53a624d46842a9444e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.673234 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 80b11a6fd84840178b81d2b7610c7351 (DEFAULT_TABLE table=default.Table_acbd237bfa684410a9a16aec1821e400 [id=84e44915fea341c2b33012883a1b3ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.674324 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b11a6fd84840178b81d2b7610c7351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.674199 17815 raft_consensus.cc:482] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.674676 17815 raft_consensus.cc:504] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.621611 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 6e20fc94eecf4a14840ca88177070e33 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.656842 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 80b11a6fd84840178b81d2b7610c7351 (DEFAULT_TABLE table=default.Table_acbd237bfa684410a9a16aec1821e400 [id=84e44915fea341c2b33012883a1b3ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.676594 17815 leader_election.cc:288] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:24.680117 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.683972 17755 raft_consensus.cc:504] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:24.684533 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a2e5bc4ae186404c896814b95d6cc7ef consensus state has no leader
I20260923 17:31:24.685608 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.587736 17144 tablet_bootstrap.cc:654] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.693454 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.694512 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feebaa76bcd4ff5b707814191cb3341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.692792 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 205) took 1626 ms. Trace:
I20260923 17:31:24.700387 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0c75099d4d43339e88bf7527b10af9" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:24.700966 17039 raft_consensus.cc:3037] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.699949 17403 rpcz_store.cc:276] 0923 17:31:23.066197 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.066284 (+ 87us) service_pool.cc:227] Handling call
0923 17:31:24.175199 (+1108915us) ts_tablet_manager.cc:1145] Deleting tablet f1ccdf1bd7a64f51965e0a6dc7e033c9
0923 17:31:24.175238 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.301984 (+126746us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.571003 (+269019us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.692731 (+121728us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1108455}
I20260923 17:31:24.705703 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (table default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]) successfully deleted
I20260923 17:31:24.710234 17039 raft_consensus.cc:2454] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:24.711006 17403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 206) took 1644 ms. Trace:
I20260923 17:31:24.711292 17403 rpcz_store.cc:276] 0923 17:31:23.067021 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.067786 (+ 765us) service_pool.cc:227] Handling call
0923 17:31:24.708036 (+1640250us) ts_tablet_manager.cc:1145] Deleting tablet eb2beb8360d54efeb2fa74c005d011d1
0923 17:31:24.708076 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.710951 (+ 2875us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1639726}
W20260923 17:31:24.712432 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because the tablet was not found. No further retry: Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:24.713060 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e20fc94eecf4a14840ca88177070e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.716732 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b11a6fd84840178b81d2b7610c7351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.717355 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 354f66c852b94b1d9f44b6bc012cf81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:24.723668 17755 leader_election.cc:288] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:24.728206 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 8e7c8e360d954ce78913e78c8662c341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e7c8e360d954ce78913e78c8662c341 already in progress: creating tablet
I20260923 17:31:24.729759 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:24.730309 17039 raft_consensus.cc:3037] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.731041 17144 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:24.731952 17144 ts_tablet_manager.cc:1397] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.183s user 0.009s sys 0.001s
I20260923 17:31:24.734035 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet a86f4b7ee14948f0989af54b3b77dc71 (table default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]) successfully deleted
I20260923 17:31:24.732635 17403 tablet_replica.cc:330] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:24.743677 17403 raft_consensus.cc:2229] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:24.744747 17542 ts_tablet_manager.cc:1918] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:24.745134 17542 log.cc:1196] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:24.745831 17542 ts_tablet_manager.cc:1939] T 2c842eff86634bf4bc70c426b5226d66 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:24.747206 17810 tablet_replica.cc:330] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:24.747910 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 184) took 1607 ms. Trace:
I20260923 17:31:24.748553 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:24.684118 16972 leader_election.cc:341] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.750304 17800 raft_consensus.cc:482] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.750676 17012 tablet_service.cc:1467] Processing CreateTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:24.750708 17800 raft_consensus.cc:504] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.751765 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feebaa76bcd4ff5b707814191cb3341. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:24.751207 16677 leader_election.cc:341] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.752666 17800 leader_election.cc:288] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.754691 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.754763 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bec348cce841a689f69bacdf21004d" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:24.755530 16739 raft_consensus.cc:2379] T 14bec348cce841a689f69bacdf21004d P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate e5c071cdf65042a59cad226a15a5532c in this term.
I20260923 17:31:24.689049 17146 raft_consensus.cc:348] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.756325 16826 leader_election.cc:302] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:24.756587 17146 raft_consensus.cc:374] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.756865 17146 raft_consensus.cc:729] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.757555 17800 raft_consensus.cc:3037] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.757556 17146 consensus_queue.cc:260] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.598826 17679 raft_consensus.cc:504] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.762301 17679 leader_election.cc:288] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.767627 17800 raft_consensus.cc:2726] T 14bec348cce841a689f69bacdf21004d P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.767558 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:24.768098 17040 raft_consensus.cc:3037] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.778580 17040 raft_consensus.cc:2454] T e4a2a0d1f4344ad69eb2c0a4e208839a P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:24.783877 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:24.785178 16677 leader_election.cc:341] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.748193 17542 rpcz_store.cc:276] 0923 17:31:23.140528 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.140630 (+ 102us) service_pool.cc:227] Handling call
0923 17:31:24.601507 (+1460877us) ts_tablet_manager.cc:1145] Deleting tablet 2c842eff86634bf4bc70c426b5226d66
0923 17:31:24.601546 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.684651 (+ 83105us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.744475 (+ 59824us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:24.747848 (+ 3373us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1460298}
I20260923 17:31:24.790390 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 2c842eff86634bf4bc70c426b5226d66 (table default.Table_0cef567b57404c0ab86a30bf79964ef2 [id=a548e994ee5243b4afad47596f1b4a4e]) successfully deleted
I20260923 17:31:24.793211 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:24.792371 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 183) took 1652 ms. Trace:
I20260923 17:31:24.794559 16383 catalog_manager.cc:6565] ReplaceTablet: tablet c8895cffac5945a19a5de7fba1319f22 deleted and replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a
I20260923 17:31:24.794351 17542 rpcz_store.cc:276] 0923 17:31:23.139812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.149620 (+ 9808us) service_pool.cc:227] Handling call
0923 17:31:24.792049 (+1642429us) ts_tablet_manager.cc:1145] Deleting tablet a80034cbf6354e1bbe0a0220e43251c3
0923 17:31:24.792095 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.792311 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1641740}
W20260923 17:31:24.749936 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 65848.7us
W20260923 17:31:24.797003 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a80034cbf6354e1bbe0a0220e43251c3 because the tablet was not found. No further retry: Not found: Tablet not found: a80034cbf6354e1bbe0a0220e43251c3
W20260923 17:31:24.797351 16972 leader_election.cc:341] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.797713 16972 leader_election.cc:302] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.798328 17808 raft_consensus.cc:2726] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.798337 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:24.799324 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:24.801852 16678 leader_election.cc:302] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
W20260923 17:31:24.805313 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 8e7c8e360d954ce78913e78c8662c341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e7c8e360d954ce78913e78c8662c341 already in progress: creating tablet
I20260923 17:31:24.807093 17542 tablet_replica.cc:330] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:24.808789 17810 raft_consensus.cc:2229] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:24.809113 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet d20c9b997a024151a810ea015c20e18f (table default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c]). Expected version 1 got 0
I20260923 17:31:24.809402 17810 raft_consensus.cc:2258] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.810523 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e2fa2e2d002746c8ba2b0dcf45372aa8 (table default.Table_dcf1d5122d0c421bab95f4f6e66d3f47 [id=6d073df311674a358a520dd1586a3345]). Expected version 3 got 2
I20260923 17:31:24.811192 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:24.813890 17403 pending_rounds.cc:62] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:24.814237 17403 pending_rounds.cc:68] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596468244660224 op_type: NO_OP noop_request { }
I20260923 17:31:24.814863 17403 raft_consensus.cc:2866] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:24.815418 17403 raft_consensus.cc:2258] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.815862 17039 raft_consensus.cc:2454] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:24.811751 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:24.827018 17143 tablet_bootstrap.cc:492] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.827476 17143 ts_tablet_manager.cc:1397] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.239s user 0.024s sys 0.006s
I20260923 17:31:24.827158 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:24.828797 16387 catalog_manager.cc:5495] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:24.830470 17143 raft_consensus.cc:348] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.818296 17755 raft_consensus.cc:2781] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.831631 17755 raft_consensus.cc:686] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:24.815866 17144 raft_consensus.cc:348] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.830667 16387 catalog_manager.cc:5495] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:24.832899 17144 raft_consensus.cc:374] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.833158 17144 raft_consensus.cc:729] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.833948 17144 consensus_queue.cc:260] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.844261 17403 ts_tablet_manager.cc:1905] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.848760 17143 raft_consensus.cc:374] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.848855 16886 raft_consensus.cc:3037] T 191292e60d0d46fb89b52390feb48f80 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.849082 17143 raft_consensus.cc:729] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.834741 17542 raft_consensus.cc:2229] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:24.850509 17542 raft_consensus.cc:2258] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.852926 17542 ts_tablet_manager.cc:1905] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.849623 17143 consensus_queue.cc:260] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.855149 17143 ts_tablet_manager.cc:1428] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.027s user 0.001s sys 0.004s
I20260923 17:31:24.855911 17143 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:24.857173 17144 ts_tablet_manager.cc:1428] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.122s user 0.005s sys 0.000s
I20260923 17:31:24.859782 17144 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:24.860553 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:24.832468 17755 consensus_queue.cc:237] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.865777 16678 leader_election.cc:302] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
W20260923 17:31:24.868741 16826 consensus_peers.cc:487] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:24.817127 17810 ts_tablet_manager.cc:1905] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.834813 17808 raft_consensus.cc:482] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.870623 17808 raft_consensus.cc:504] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.845702 17146 ts_tablet_manager.cc:1428] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.181s user 0.005s sys 0.001s
I20260923 17:31:24.872452 17808 leader_election.cc:288] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:24.872900 17679 raft_consensus.cc:482] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.873490 17679 raft_consensus.cc:504] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.874771 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.875584 17679 leader_election.cc:288] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.850901 16388 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 57e3ed103222488c8dfb9e474358100d
W20260923 17:31:24.876406 16972 leader_election.cc:341] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.877091 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:24.878394 16972 leader_election.cc:341] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.878623 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:24.878724 16972 leader_election.cc:302] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.879163 17035 raft_consensus.cc:2454] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:24.879981 17800 raft_consensus.cc:482] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:24.880388 17800 raft_consensus.cc:504] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.853466 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_f833f76b0fe74b038732663e9c002a53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:24.882411 17800 leader_election.cc:288] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.882849 17808 raft_consensus.cc:2726] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.883630 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
W20260923 17:31:24.884006 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f833f76b0fe74b038732663e9c002a53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.886940 17144 tablet_bootstrap.cc:654] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.888406 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:24.888602 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:24.889380 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "373ad39e4d134ffc80caa73a83b80795" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:24.889549 16387 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:24.892984 17840 raft_consensus.cc:2781] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.893412 17840 raft_consensus.cc:686] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:24.894132 17840 consensus_queue.cc:237] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:24.899258 16677 leader_election.cc:341] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.899242 16826 leader_election.cc:302] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:24.900171 17800 raft_consensus.cc:2726] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:24.903692 16678 leader_election.cc:302] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.906638 17146 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:24.909830 17834 raft_consensus.cc:2781] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:24.910156 17834 raft_consensus.cc:482] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:24.910432 17834 raft_consensus.cc:3037] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.927627 17834 raft_consensus.cc:504] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.931232 17143 tablet_bootstrap.cc:654] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.943280 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:24.944649 16677 leader_election.cc:341] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:24.946736 17834 leader_election.cc:288] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:24.946859 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e329dd46a3a245b6826b01ac8de56621" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:24.947520 17039 raft_consensus.cc:3037] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:24.949996 17143 tablet_bootstrap.cc:492] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:24.950392 17143 ts_tablet_manager.cc:1397] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.000s
I20260923 17:31:24.952654 17143 raft_consensus.cc:348] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.953253 17143 raft_consensus.cc:374] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:24.953501 17143 raft_consensus.cc:729] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:24.957340 17039 raft_consensus.cc:2454] T e329dd46a3a245b6826b01ac8de56621 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:24.954026 17143 consensus_queue.cc:260] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:24.960850 17143 ts_tablet_manager.cc:1428] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260923 17:31:24.962716 17143 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:24.968174 17143 tablet_bootstrap.cc:654] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:24.958791 17810 ts_tablet_manager.cc:1918] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:24.969213 17810 log.cc:1196] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/1e0c75099d4d43339e88bf7527b10af9
I20260923 17:31:24.969863 17810 ts_tablet_manager.cc:1939] T 1e0c75099d4d43339e88bf7527b10af9 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:24.973592 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 1e0c75099d4d43339e88bf7527b10af9 (table default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]) successfully deleted
I20260923 17:31:24.974686 17810 tablet_replica.cc:330] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:24.975627 17810 raft_consensus.cc:2229] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:24.976281 17810 raft_consensus.cc:2258] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:24.978772 17810 ts_tablet_manager.cc:1905] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:24.986333 16678 leader_election.cc:302] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:24.987742 17843 raft_consensus.cc:2781] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:24.996163 16383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_14b0bd687f5346b1b56e3bc95c7437ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:24.997850 16383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14b0bd687f5346b1b56e3bc95c7437ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:24.995748 17088 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:25.001025 17144 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.001473 17144 ts_tablet_manager.cc:1397] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.142s user 0.010s sys 0.000s
I20260923 17:31:24.988297 17843 raft_consensus.cc:686] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:25.003947 17144 raft_consensus.cc:348] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.004758 17144 raft_consensus.cc:374] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.005010 17144 raft_consensus.cc:729] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.005697 17144 consensus_queue.cc:260] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.008009 17144 ts_tablet_manager.cc:1428] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 17:31:25.007161 17146 tablet_bootstrap.cc:654] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.008860 17144 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.002413 17843 consensus_queue.cc:237] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.017077 17144 tablet_bootstrap.cc:654] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.017571 17403 ts_tablet_manager.cc:1918] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:25.017930 17403 log.cc:1196] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/1e0c75099d4d43339e88bf7527b10af9
I20260923 17:31:25.018541 17403 ts_tablet_manager.cc:1939] T 1e0c75099d4d43339e88bf7527b10af9 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:25.019665 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 1e0c75099d4d43339e88bf7527b10af9 (table default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]) successfully deleted
I20260923 17:31:25.033732 17144 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.034209 17144 ts_tablet_manager.cc:1397] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20260923 17:31:25.034269 17403 tablet_replica.cc:330] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:25.035092 17403 raft_consensus.cc:2229] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:25.035514 17403 raft_consensus.cc:2258] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:25.040428 17144 raft_consensus.cc:348] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.041165 17144 raft_consensus.cc:374] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.041416 17144 raft_consensus.cc:729] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.048059 17403 ts_tablet_manager.cc:1905] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:25.049510 17143 tablet_bootstrap.cc:492] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:25.049888 17143 ts_tablet_manager.cc:1397] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.087s user 0.035s sys 0.018s
I20260923 17:31:25.069698 17810 ts_tablet_manager.cc:1918] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:25.070102 17810 log.cc:1196] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:25.070771 17810 ts_tablet_manager.cc:1939] T c8895cffac5945a19a5de7fba1319f22 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:25.082402 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]) successfully deleted
I20260923 17:31:25.084426 17403 ts_tablet_manager.cc:1918] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:25.084761 17403 log.cc:1196] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/a86f4b7ee14948f0989af54b3b77dc71
I20260923 17:31:25.085387 17403 ts_tablet_manager.cc:1939] T a86f4b7ee14948f0989af54b3b77dc71 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:25.086704 17403 tablet_replica.cc:330] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:25.087100 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet a86f4b7ee14948f0989af54b3b77dc71 (table default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]) successfully deleted
I20260923 17:31:25.087342 17403 raft_consensus.cc:2229] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:25.088285 17403 raft_consensus.cc:2258] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:25.090018 17403 ts_tablet_manager.cc:1905] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:25.101686 17144 consensus_queue.cc:260] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.105036 17403 ts_tablet_manager.cc:1918] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:25.105406 17403 log.cc:1196] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:25.106073 17403 ts_tablet_manager.cc:1939] T c8895cffac5945a19a5de7fba1319f22 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:25.108043 17144 ts_tablet_manager.cc:1428] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.074s user 0.007s sys 0.000s
I20260923 17:31:25.112270 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]) successfully deleted
I20260923 17:31:25.121531 17144 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.134299 17146 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.134789 17146 ts_tablet_manager.cc:1397] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.228s user 0.005s sys 0.005s
I20260923 17:31:25.137135 17144 tablet_bootstrap.cc:654] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.145609 17146 raft_consensus.cc:348] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.146699 17146 raft_consensus.cc:374] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.146960 17146 raft_consensus.cc:729] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.147703 17146 consensus_queue.cc:260] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.149984 17146 ts_tablet_manager.cc:1428] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
W20260923 17:31:25.177549 17244 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 0.684s user 0.007s sys 0.003s
W20260923 17:31:25.177968 17244 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 0.685s user 0.007s sys 0.003s
I20260923 17:31:25.191565 17146 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:25.203984 17808 consensus_queue.cc:1035] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:25.052112 17143 raft_consensus.cc:348] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.227417 17143 raft_consensus.cc:374] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.227921 17143 raft_consensus.cc:729] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.230710 17542 ts_tablet_manager.cc:1918] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:25.231359 17542 log.cc:1196] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/1e0c75099d4d43339e88bf7527b10af9
I20260923 17:31:25.229147 17143 consensus_queue.cc:260] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.233690 17797 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.234305 17797 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.245247 17815 consensus_queue.cc:1035] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:25.247359 16826 consensus_peers.cc:487] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a86f4b7ee14948f0989af54b3b77dc71. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:25.248404 17542 ts_tablet_manager.cc:1939] T 1e0c75099d4d43339e88bf7527b10af9 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:25.252883 17797 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.257872 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 187) took 1028 ms. Trace:
I20260923 17:31:25.258302 17542 rpcz_store.cc:276] 0923 17:31:24.228974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.229728 (+ 754us) service_pool.cc:227] Handling call
0923 17:31:24.806936 (+577208us) ts_tablet_manager.cc:1145] Deleting tablet 1e0c75099d4d43339e88bf7527b10af9
0923 17:31:24.806975 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:24.951539 (+144564us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.230126 (+278587us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.257767 (+ 27641us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23896,"tablet-delete.queue_time_us":576722}
I20260923 17:31:25.261284 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 1e0c75099d4d43339e88bf7527b10af9 (table default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]) successfully deleted
I20260923 17:31:25.266373 17542 tablet_replica.cc:330] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:25.267573 16826 consensus_peers.cc:487] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:25.267784 17815 consensus_queue.cc:1035] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:25.268330 16826 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:25.277523 17144 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.277980 17144 ts_tablet_manager.cc:1397] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.157s user 0.043s sys 0.035s
I20260923 17:31:25.280388 17144 raft_consensus.cc:348] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.281090 17144 raft_consensus.cc:374] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.281350 17144 raft_consensus.cc:729] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.281910 17144 consensus_queue.cc:260] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.285712 17144 ts_tablet_manager.cc:1428] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260923 17:31:25.286518 17144 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:25.303422 16972 consensus_peers.cc:487] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:25.304273 17858 consensus_queue.cc:1035] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260923 17:31:25.306211 17144 tablet_bootstrap.cc:654] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.314541 17146 tablet_bootstrap.cc:654] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.332582 17542 raft_consensus.cc:2229] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:25.333945 17542 raft_consensus.cc:2258] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:25.336490 17542 ts_tablet_manager.cc:1905] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:25.341240 16373 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.188:46075, user_credentials={real_user=slave}} blocked reactor thread for 44335.3us
W20260923 17:31:25.323855 16827 consensus_peers.cc:487] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 051f2e478a324df98647d90107d55533. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:25.344828 16827 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 46705.5us
W20260923 17:31:25.347404 16827 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:25.347842 16827 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:25.348613 17797 raft_consensus.cc:2726] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:25.350842 17143 ts_tablet_manager.cc:1428] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.301s user 0.014s sys 0.000s
I20260923 17:31:25.356079 17143 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:25.363938 17143 tablet_bootstrap.cc:654] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.368626 17146 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.369069 17146 ts_tablet_manager.cc:1397] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.178s user 0.010s sys 0.000s
I20260923 17:31:25.371363 17146 raft_consensus.cc:348] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.372037 17144 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.372483 17144 ts_tablet_manager.cc:1397] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.086s user 0.031s sys 0.006s
I20260923 17:31:25.374632 17144 raft_consensus.cc:348] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.375285 17144 raft_consensus.cc:374] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.375520 17144 raft_consensus.cc:729] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.376390 17144 consensus_queue.cc:260] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.397574 17144 ts_tablet_manager.cc:1428] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.025s user 0.003s sys 0.002s
I20260923 17:31:25.400161 17867 raft_consensus.cc:482] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.400688 17867 raft_consensus.cc:504] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.402712 17867 leader_election.cc:288] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:25.403987 17144 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.406036 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:25.407249 16972 leader_election.cc:341] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.372069 17146 raft_consensus.cc:374] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.407780 17146 raft_consensus.cc:729] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.408439 17146 consensus_queue.cc:260] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.410374 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.411362 17143 tablet_bootstrap.cc:492] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:25.412086 17143 ts_tablet_manager.cc:1397] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.056s user 0.022s sys 0.023s
W20260923 17:31:25.411821 16972 leader_election.cc:341] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.414553 16972 leader_election.cc:302] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:25.415196 17815 raft_consensus.cc:2726] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:25.417760 17146 ts_tablet_manager.cc:1428] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260923 17:31:25.424613 17143 raft_consensus.cc:348] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.425343 17143 raft_consensus.cc:374] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.425580 17143 raft_consensus.cc:729] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.427824 17144 tablet_bootstrap.cc:654] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.432565 17244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:25.435024 17244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:25.435611 17244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:25.437081 17146 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:25.442654 17146 tablet_bootstrap.cc:654] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.446084 17867 raft_consensus.cc:482] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.446511 17867 raft_consensus.cc:504] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.448264 17867 leader_election.cc:288] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:25.450053 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.450569 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:25.451560 16972 leader_election.cc:341] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:25.452446 16972 leader_election.cc:341] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.452771 16972 leader_election.cc:302] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:25.453584 17867 raft_consensus.cc:2726] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:25.456337 17143 consensus_queue.cc:260] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.458716 17143 ts_tablet_manager.cc:1428] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.046s user 0.004s sys 0.000s
I20260923 17:31:25.472908 17143 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:25.477439 17144 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.477914 17144 ts_tablet_manager.cc:1397] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.000s
I20260923 17:31:25.482276 17144 raft_consensus.cc:348] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.492883 17144 raft_consensus.cc:374] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.493158 17144 raft_consensus.cc:729] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.495874 17146 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.496757 17143 tablet_bootstrap.cc:654] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.499680 17542 ts_tablet_manager.cc:1918] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:25.500043 17542 log.cc:1196] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a86f4b7ee14948f0989af54b3b77dc71
I20260923 17:31:25.500337 17146 ts_tablet_manager.cc:1397] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.008s
I20260923 17:31:25.500161 17144 consensus_queue.cc:260] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.500746 17542 ts_tablet_manager.cc:1939] T a86f4b7ee14948f0989af54b3b77dc71 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:25.503051 17146 raft_consensus.cc:348] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.505211 17144 ts_tablet_manager.cc:1428] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260923 17:31:25.505517 17146 raft_consensus.cc:374] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.505796 17146 raft_consensus.cc:729] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.506480 17146 consensus_queue.cc:260] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.511483 17144 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.515060 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 188) took 1278 ms. Trace:
I20260923 17:31:25.523694 17146 ts_tablet_manager.cc:1428] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.023s user 0.004s sys 0.001s
I20260923 17:31:25.515400 17542 rpcz_store.cc:276] 0923 17:31:24.236877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.237607 (+ 730us) service_pool.cc:227] Handling call
0923 17:31:25.266102 (+1028495us) ts_tablet_manager.cc:1145] Deleting tablet a86f4b7ee14948f0989af54b3b77dc71
0923 17:31:25.266204 (+ 102us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:25.394728 (+128524us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.499428 (+104700us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.514880 (+ 15452us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":64886,"tablet-delete.queue_time_us":1027855}
I20260923 17:31:25.539827 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a86f4b7ee14948f0989af54b3b77dc71 (table default.Table_ba9a40693fbc40cda7c43b3b9a5405f0 [id=ef85ff29b8c747abb46e2271b142ef0d]) successfully deleted
I20260923 17:31:25.544961 17144 tablet_bootstrap.cc:654] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.547170 17858 raft_consensus.cc:482] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.548277 17858 raft_consensus.cc:504] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.550287 17858 leader_election.cc:288] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.551946 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:25.540202 17542 tablet_replica.cc:330] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:25.553745 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208c90d96f349988acb6705f2f438d5" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.554397 17040 raft_consensus.cc:2379] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:25.554432 17542 raft_consensus.cc:2229] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:25.555049 17542 raft_consensus.cc:2258] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:25.555548 16827 leader_election.cc:302] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:25.547734 17143 tablet_bootstrap.cc:492] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:25.556740 17858 raft_consensus.cc:3037] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.556747 17143 ts_tablet_manager.cc:1397] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.000s
I20260923 17:31:25.562674 17143 raft_consensus.cc:348] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.563357 17143 raft_consensus.cc:374] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.563598 17143 raft_consensus.cc:729] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.565730 17146 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:25.567245 17843 raft_consensus.cc:482] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.567718 17843 raft_consensus.cc:504] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.569677 17843 leader_election.cc:288] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.571117 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.571679 17040 raft_consensus.cc:2454] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.565197 17858 raft_consensus.cc:2726] T 3208c90d96f349988acb6705f2f438d5 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:25.571121 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.572775 16678 leader_election.cc:302] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:25.573920 17146 tablet_bootstrap.cc:654] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:25.573969 16677 leader_election.cc:341] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.574201 17834 raft_consensus.cc:2781] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:25.574502 17834 raft_consensus.cc:482] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:25.574780 17834 raft_consensus.cc:3037] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.568720 17542 ts_tablet_manager.cc:1905] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:25.564213 17143 consensus_queue.cc:260] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.580842 17143 ts_tablet_manager.cc:1428] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260923 17:31:25.594746 17144 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.595202 17144 ts_tablet_manager.cc:1397] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.000s
I20260923 17:31:25.601819 16886 raft_consensus.cc:3037] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.604388 17143 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:25.616431 17867 consensus_queue.cc:1035] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:25.625483 16864 tablet_service.cc:1467] Processing CreateTablet for tablet c3d4abafcef943c08ae0755bae06f32c (DEFAULT_TABLE table=default.Table_566b62c66bac46eb97bbae9720393eec [id=bda2189f4f514e1da3b6f0a226731385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.626931 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d4abafcef943c08ae0755bae06f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.629746 17753 raft_consensus.cc:482] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.630163 17753 raft_consensus.cc:504] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.632144 17753 leader_election.cc:288] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.630303 17144 raft_consensus.cc:348] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.633559 17834 raft_consensus.cc:504] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.634050 17144 raft_consensus.cc:374] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.635473 17834 leader_election.cc:288] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.636443 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:25.636623 16717 tablet_service.cc:1467] Processing CreateTablet for tablet c3d4abafcef943c08ae0755bae06f32c (DEFAULT_TABLE table=default.Table_566b62c66bac46eb97bbae9720393eec [id=bda2189f4f514e1da3b6f0a226731385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.637025 17040 raft_consensus.cc:3037] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.637620 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d4abafcef943c08ae0755bae06f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.634366 17144 raft_consensus.cc:729] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.641932 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.642616 17035 raft_consensus.cc:2454] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.640669 17144 consensus_queue.cc:260] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.642855 17040 raft_consensus.cc:2454] T 7efa62885c564a9bb1ca1b9beacf6e73 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:25.643523 17867 consensus_queue.cc:1035] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:25.644318 17144 ts_tablet_manager.cc:1428] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.049s user 0.007s sys 0.000s
I20260923 17:31:25.652938 17144 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.656822 17834 raft_consensus.cc:482] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.657253 17834 raft_consensus.cc:504] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.658946 17834 leader_election.cc:288] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.659228 17011 tablet_service.cc:1467] Processing CreateTablet for tablet c3d4abafcef943c08ae0755bae06f32c (DEFAULT_TABLE table=default.Table_566b62c66bac46eb97bbae9720393eec [id=bda2189f4f514e1da3b6f0a226731385]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.661285 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d4abafcef943c08ae0755bae06f32c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.662313 17146 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.662755 17146 ts_tablet_manager.cc:1397] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.015s
I20260923 17:31:25.664733 17146 raft_consensus.cc:348] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.665344 17146 raft_consensus.cc:374] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.665627 17146 raft_consensus.cc:729] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.666850 17146 consensus_queue.cc:260] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.675441 17144 tablet_bootstrap.cc:654] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.680474 16678 leader_election.cc:302] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:25.680912 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0b13a93d4445bebd43690ef36452dd (DEFAULT_TABLE table=default.Table_022c670c0e8f41a4ae4d26c8a376a76b [id=63e0c2214e5c4f2db5c0af90c12cae3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.681754 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.682086 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0b13a93d4445bebd43690ef36452dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.682369 17040 raft_consensus.cc:2454] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.693056 17146 ts_tablet_manager.cc:1428] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260923 17:31:25.692442 17144 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.695842 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0b13a93d4445bebd43690ef36452dd (DEFAULT_TABLE table=default.Table_022c670c0e8f41a4ae4d26c8a376a76b [id=63e0c2214e5c4f2db5c0af90c12cae3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.702950 16678 leader_election.cc:302] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:25.703159 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0b13a93d4445bebd43690ef36452dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.703781 17144 ts_tablet_manager.cc:1397] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
I20260923 17:31:25.703861 17843 raft_consensus.cc:2781] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:25.704175 17843 raft_consensus.cc:482] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:25.704541 17843 raft_consensus.cc:3037] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.706023 17144 raft_consensus.cc:348] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.706653 17144 raft_consensus.cc:374] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.706900 17144 raft_consensus.cc:729] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.707487 17144 consensus_queue.cc:260] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.711308 17144 ts_tablet_manager.cc:1428] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260923 17:31:25.711266 17843 raft_consensus.cc:504] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.713052 17144 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:25.720438 17843 leader_election.cc:288] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.726241 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 070d38df3f0f43edab3826835c9503ce (DEFAULT_TABLE table=default.Table_fb7c50f5b9da47e98737dbf4f90005eb [id=ef4aea9a69ae43e48cf913c554fcdac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.727392 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070d38df3f0f43edab3826835c9503ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.729038 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0b13a93d4445bebd43690ef36452dd (DEFAULT_TABLE table=default.Table_022c670c0e8f41a4ae4d26c8a376a76b [id=63e0c2214e5c4f2db5c0af90c12cae3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.725675 17753 raft_consensus.cc:2781] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:25.740752 17753 raft_consensus.cc:686] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:25.741719 17753 consensus_queue.cc:237] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.726440 17144 tablet_bootstrap.cc:654] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.726540 16678 leader_election.cc:302] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:25.756557 17843 raft_consensus.cc:2781] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:25.756953 17843 raft_consensus.cc:482] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:25.757242 17843 raft_consensus.cc:3037] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:25.757295 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a2e5bc4ae186404c896814b95d6cc7ef consensus state has no leader
I20260923 17:31:25.758412 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 070d38df3f0f43edab3826835c9503ce (DEFAULT_TABLE table=default.Table_fb7c50f5b9da47e98737dbf4f90005eb [id=ef4aea9a69ae43e48cf913c554fcdac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.759370 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070d38df3f0f43edab3826835c9503ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.759552 17146 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:25.760027 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0b13a93d4445bebd43690ef36452dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.763688 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:25.765689 17146 tablet_bootstrap.cc:654] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.766999 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:25.767522 17040 raft_consensus.cc:3037] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.773293 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 070d38df3f0f43edab3826835c9503ce (DEFAULT_TABLE table=default.Table_fb7c50f5b9da47e98737dbf4f90005eb [id=ef4aea9a69ae43e48cf913c554fcdac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.774583 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070d38df3f0f43edab3826835c9503ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.785805 17040 raft_consensus.cc:2454] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:25.794507 17144 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:25.795017 17144 ts_tablet_manager.cc:1397] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.001s
I20260923 17:31:25.798053 17146 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.798543 17146 ts_tablet_manager.cc:1397] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.002s
I20260923 17:31:25.801075 17146 raft_consensus.cc:348] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.801791 17146 raft_consensus.cc:374] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.802068 17146 raft_consensus.cc:729] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.802721 17146 consensus_queue.cc:260] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.805383 17146 ts_tablet_manager.cc:1428] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260923 17:31:25.806463 17146 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:25.825350 17843 raft_consensus.cc:504] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.831773 17015 tablet_service.cc:1467] Processing CreateTablet for tablet e54f76e45d254086983cf6ee21484063 (DEFAULT_TABLE table=default.Table_0655dcdf684c4131bc4d7118aca89c9c [id=79d78e54697e4bfd9b96251b9150336d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.831888 16720 tablet_service.cc:1467] Processing CreateTablet for tablet e54f76e45d254086983cf6ee21484063 (DEFAULT_TABLE table=default.Table_0655dcdf684c4131bc4d7118aca89c9c [id=79d78e54697e4bfd9b96251b9150336d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.832896 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54f76e45d254086983cf6ee21484063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.833057 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54f76e45d254086983cf6ee21484063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.833803 17843 leader_election.cc:288] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.838035 17146 tablet_bootstrap.cc:654] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.842370 17843 raft_consensus.cc:482] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.842852 17843 raft_consensus.cc:504] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.845117 17843 leader_election.cc:288] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.849740 16870 tablet_service.cc:1467] Processing CreateTablet for tablet e54f76e45d254086983cf6ee21484063 (DEFAULT_TABLE table=default.Table_0655dcdf684c4131bc4d7118aca89c9c [id=79d78e54697e4bfd9b96251b9150336d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.849481 16379 catalog_manager.cc:5495] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:25.850838 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54f76e45d254086983cf6ee21484063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.851465 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:25.858901 17144 raft_consensus.cc:348] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.860361 17144 raft_consensus.cc:374] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.867220 16869 tablet_service.cc:1467] Processing CreateTablet for tablet b7b4d39a636e44e68ef2229006551ffa (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.868278 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b4d39a636e44e68ef2229006551ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.870177 17146 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:25.870527 17146 ts_tablet_manager.cc:1397] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.005s
I20260923 17:31:25.872972 17146 raft_consensus.cc:348] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.873631 17146 raft_consensus.cc:374] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:25.873869 17146 raft_consensus.cc:729] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.874835 17146 consensus_queue.cc:260] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.876058 17018 tablet_service.cc:1467] Processing CreateTablet for tablet b7b4d39a636e44e68ef2229006551ffa (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.878516 17146 ts_tablet_manager.cc:1428] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260923 17:31:25.879491 17143 tablet_bootstrap.cc:654] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:25.881426 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.883108 16379 catalog_manager.cc:5495] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:25.889446 16886 raft_consensus.cc:3037] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.872586 16726 tablet_service.cc:1467] Processing CreateTablet for tablet b7b4d39a636e44e68ef2229006551ffa (DEFAULT_TABLE table=default.Table_b33b5b1e1e084f889366e77bd5fa0871 [id=7b2870f2566346ab961116970c63dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.891073 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b4d39a636e44e68ef2229006551ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.891980 16888 raft_consensus.cc:3037] T 373ad39e4d134ffc80caa73a83b80795 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.873383 17843 raft_consensus.cc:482] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.896445 17843 raft_consensus.cc:504] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.896737 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.898463 17843 leader_election.cc:288] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.898447 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:25.900622 17858 raft_consensus.cc:482] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.901048 17858 raft_consensus.cc:504] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.902928 17858 leader_election.cc:288] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:25.904387 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:25.876711 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 8a992ad5280444dda64c1325da92a707 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.909658 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a992ad5280444dda64c1325da92a707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.842824 16678 leader_election.cc:302] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:25.911221 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 68484.6us
I20260923 17:31:25.912530 17843 raft_consensus.cc:2781] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:25.912904 17867 raft_consensus.cc:482] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.913357 17867 raft_consensus.cc:504] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.913308 17843 raft_consensus.cc:686] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:25.915153 17867 leader_election.cc:288] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:25.915614 17843 consensus_queue.cc:237] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.917544 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:25.919247 16972 leader_election.cc:341] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.921296 16883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:25.922034 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.922739 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.923851 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:25.925839 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa62885c564a9bb1ca1b9beacf6e73" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:25.927439 16972 leader_election.cc:341] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.927809 16972 leader_election.cc:302] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:25.931819 17808 raft_consensus.cc:2726] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:25.944324 17144 raft_consensus.cc:729] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:25.945215 17144 consensus_queue.cc:260] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:25.947700 17144 ts_tablet_manager.cc:1428] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.152s user 0.006s sys 0.001s
I20260923 17:31:25.948601 17144 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:25.952912 16677 leader_election.cc:341] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:25.953811 16677 leader_election.cc:341] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:25.954649 16677 consensus_peers.cc:487] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:25.955341 16677 leader_election.cc:341] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.963433 17542 ts_tablet_manager.cc:1918] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:25.963840 17884 consensus_queue.cc:1035] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:25.968164 16677 leader_election.cc:341] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.970072 17834 raft_consensus.cc:482] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:25.971503 17542 log.cc:1196] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:25.970556 17834 raft_consensus.cc:504] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:25.974771 16677 leader_election.cc:341] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.980654 17542 ts_tablet_manager.cc:1939] T c8895cffac5945a19a5de7fba1319f22 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:25.982390 17012 tablet_service.cc:1467] Processing CreateTablet for tablet 8a992ad5280444dda64c1325da92a707 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.984714 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8fc3755f2645289656aef924ac6f38 (DEFAULT_TABLE table=default.Table_6974434dff19466482f1dccefd5689a8 [id=89f245ee69524e509af8d4b9c5c9a74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.986457 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 8a992ad5280444dda64c1325da92a707 (DEFAULT_TABLE table=default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.986837 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151184083ebb4ae4bf24fe171d33cfab" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.987416 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a992ad5280444dda64c1325da92a707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.988816 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a90f7348a964a7f871e93dde7ab82f2" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:25.989277 17041 raft_consensus.cc:3037] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:25.990957 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8fc3755f2645289656aef924ac6f38 (DEFAULT_TABLE table=default.Table_6974434dff19466482f1dccefd5689a8 [id=89f245ee69524e509af8d4b9c5c9a74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:25.992355 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.992904 17036 raft_consensus.cc:2454] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.994243 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.994822 17038 raft_consensus.cc:2454] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.997381 17031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:25.997944 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8fc3755f2645289656aef924ac6f38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:25.997915 17031 raft_consensus.cc:2454] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:25.976368 17834 leader_election.cc:288] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:25.975402 16677 leader_election.cc:341] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:25.996006 17884 consensus_queue.cc:1035] T 3208c90d96f349988acb6705f2f438d5 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:25.999816 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
W20260923 17:31:26.001458 16677 consensus_peers.cc:487] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:26.003285 17041 raft_consensus.cc:2454] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.007337 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8fc3755f2645289656aef924ac6f38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.007759 17010 tablet_service.cc:1467] Processing CreateTablet for tablet b77ef164608f4ced8b1c2ecab6f88c1a (DEFAULT_TABLE table=default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.009362 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77ef164608f4ced8b1c2ecab6f88c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.003888 16678 leader_election.cc:302] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.016817 17884 raft_consensus.cc:2781] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.017387 17884 raft_consensus.cc:482] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.017704 17884 raft_consensus.cc:3037] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.018661 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a992ad5280444dda64c1325da92a707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.018564 17146 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.020951 16827 leader_election.cc:302] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:26.022666 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b4d39a636e44e68ef2229006551ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.023555 17146 tablet_bootstrap.cc:654] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.032259 17858 raft_consensus.cc:2726] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.034425 16865 tablet_service.cc:1467] Processing CreateTablet for tablet b77ef164608f4ced8b1c2ecab6f88c1a (DEFAULT_TABLE table=default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.035597 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77ef164608f4ced8b1c2ecab6f88c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.037078 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8fc3755f2645289656aef924ac6f38 (DEFAULT_TABLE table=default.Table_6974434dff19466482f1dccefd5689a8 [id=89f245ee69524e509af8d4b9c5c9a74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.038059 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8fc3755f2645289656aef924ac6f38. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:26.056332 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 54967.4us
W20260923 17:31:26.057099 16677 leader_election.cc:341] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.072342 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 6398fa526d9c432e8c48ebae0b4612aa (DEFAULT_TABLE table=default.Table_027bd79de4ee4fc7aee0f4c904532006 [id=efee0a7d45804f87bdbd58abd74b2ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.073494 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6398fa526d9c432e8c48ebae0b4612aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.071173 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 6398fa526d9c432e8c48ebae0b4612aa (DEFAULT_TABLE table=default.Table_027bd79de4ee4fc7aee0f4c904532006 [id=efee0a7d45804f87bdbd58abd74b2ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.074491 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 11123f6ee880405ea76479c80fcea7e7 (DEFAULT_TABLE table=default.Table_f5da2cb10b2249359ef13479fa2c524d [id=5fb6fe1d39094aaaa40fe796a662f3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.075719 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11123f6ee880405ea76479c80fcea7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.076413 17144 tablet_bootstrap.cc:654] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.076540 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6398fa526d9c432e8c48ebae0b4612aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.092672 17867 raft_consensus.cc:482] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.093230 17867 raft_consensus.cc:504] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.098184 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.099310 17867 leader_election.cc:288] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:26.100117 16972 leader_election.cc:341] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.107261 17884 raft_consensus.cc:504] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.108023 16726 tablet_service.cc:1467] Processing CreateTablet for tablet b77ef164608f4ced8b1c2ecab6f88c1a (DEFAULT_TABLE table=default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.111091 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77ef164608f4ced8b1c2ecab6f88c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.112651 17884 leader_election.cc:288] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.113188 16678 leader_election.cc:302] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.115437 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:26.116532 17146 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:26.117000 17146 ts_tablet_manager.cc:1397] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.099s user 0.013s sys 0.008s
I20260923 17:31:26.117882 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 6398fa526d9c432e8c48ebae0b4612aa (DEFAULT_TABLE table=default.Table_027bd79de4ee4fc7aee0f4c904532006 [id=efee0a7d45804f87bdbd58abd74b2ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.118775 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6398fa526d9c432e8c48ebae0b4612aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.119545 17884 raft_consensus.cc:2781] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.119877 17884 raft_consensus.cc:482] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.120169 17884 raft_consensus.cc:3037] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.123945 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:26.124759 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 11123f6ee880405ea76479c80fcea7e7 (DEFAULT_TABLE table=default.Table_f5da2cb10b2249359ef13479fa2c524d [id=5fb6fe1d39094aaaa40fe796a662f3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.125738 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11123f6ee880405ea76479c80fcea7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.126727 17146 raft_consensus.cc:348] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.127406 17146 raft_consensus.cc:374] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.127646 17146 raft_consensus.cc:729] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.135753 17867 raft_consensus.cc:482] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.135754 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.137015 16972 leader_election.cc:341] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.137398 16972 leader_election.cc:302] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.138118 17542 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 193) took 1337 ms. Trace:
I20260923 17:31:26.138330 17808 raft_consensus.cc:2726] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.136199 17867 raft_consensus.cc:504] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.138427 17542 rpcz_store.cc:276] 0923 17:31:24.800509 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.860381 (+ 59872us) service_pool.cc:227] Handling call
0923 17:31:25.539996 (+679615us) ts_tablet_manager.cc:1145] Deleting tablet c8895cffac5945a19a5de7fba1319f22
0923 17:31:25.540052 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:25.839273 (+299221us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.963203 (+123930us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:26.138066 (+174863us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":678921}
I20260923 17:31:26.140379 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet c8895cffac5945a19a5de7fba1319f22 (table default.Table_0d464342027a447fbc67d549f98de051 [id=a67f05c6ae754dcba81905a4dd293a0f]) successfully deleted
I20260923 17:31:26.142822 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.144191 16972 leader_election.cc:341] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.148250 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:26.149636 16972 leader_election.cc:341] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.150074 16972 leader_election.cc:302] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.150791 17808 raft_consensus.cc:2726] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.151405 17867 leader_election.cc:288] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:26.156687 16677 leader_election.cc:341] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:26.157312 16677 leader_election.cc:341] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.156677 17146 consensus_queue.cc:260] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.164105 17146 ts_tablet_manager.cc:1428] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.047s user 0.004s sys 0.001s
I20260923 17:31:26.165886 17144 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.166294 17144 ts_tablet_manager.cc:1397] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.218s user 0.001s sys 0.008s
I20260923 17:31:26.167587 16887 raft_consensus.cc:3037] T 151184083ebb4ae4bf24fe171d33cfab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:26.173810 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 60688.1us
I20260923 17:31:26.174402 17143 tablet_bootstrap.cc:492] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.174775 17143 ts_tablet_manager.cc:1397] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.593s user 0.003s sys 0.021s
I20260923 17:31:26.175513 17843 consensus_queue.cc:1035] T e4a2a0d1f4344ad69eb2c0a4e208839a P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:26.177743 16678 leader_election.cc:302] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.178392 17834 raft_consensus.cc:2781] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.178695 17834 raft_consensus.cc:482] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.178973 17834 raft_consensus.cc:3037] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.186892 16678 leader_election.cc:302] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.188582 17834 raft_consensus.cc:504] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.190562 17834 leader_election.cc:288] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.191543 17834 raft_consensus.cc:2781] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.198974 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 11123f6ee880405ea76479c80fcea7e7 (DEFAULT_TABLE table=default.Table_f5da2cb10b2249359ef13479fa2c524d [id=5fb6fe1d39094aaaa40fe796a662f3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.200786 17146 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.207620 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:26.217211 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11123f6ee880405ea76479c80fcea7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.220186 17867 raft_consensus.cc:482] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.220679 17867 raft_consensus.cc:504] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.223805 17867 leader_election.cc:288] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:26.224123 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.225735 16972 leader_election.cc:341] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.177069 17143 raft_consensus.cc:348] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.228688 17143 raft_consensus.cc:374] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.191921 17834 raft_consensus.cc:686] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.220474 17144 raft_consensus.cc:348] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.231563 17834 consensus_queue.cc:237] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.231734 17144 raft_consensus.cc:374] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.233044 17144 raft_consensus.cc:729] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.233711 17144 consensus_queue.cc:260] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.223453 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:26.236685 16972 leader_election.cc:341] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.237056 16972 leader_election.cc:302] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:26.212291 16677 leader_election.cc:341] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.240406 17867 raft_consensus.cc:2726] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.247103 17143 raft_consensus.cc:729] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.249887 17144 ts_tablet_manager.cc:1428] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.083s user 0.006s sys 0.001s
I20260923 17:31:26.250697 17146 tablet_bootstrap.cc:654] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.251231 17143 consensus_queue.cc:260] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.264389 17144 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.265429 17143 ts_tablet_manager.cc:1428] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.090s user 0.002s sys 0.003s
W20260923 17:31:26.266607 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 79788.9us
I20260923 17:31:26.269150 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "261d1d03290546969a74ca60beb890eb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.269775 17041 raft_consensus.cc:3037] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.271523 17143 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.259573 17884 raft_consensus.cc:504] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.274973 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:26.276058 16677 leader_election.cc:341] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.276391 17884 leader_election.cc:288] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.277275 17041 raft_consensus.cc:2454] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.286527 17144 tablet_bootstrap.cc:654] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.293895 17143 tablet_bootstrap.cc:654] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.311510 17143 tablet_bootstrap.cc:492] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.311925 17143 ts_tablet_manager.cc:1397] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.006s
I20260923 17:31:26.316915 17144 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.317461 17144 ts_tablet_manager.cc:1397] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.056s user 0.010s sys 0.002s
I20260923 17:31:26.326233 16678 leader_election.cc:302] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.327700 17884 raft_consensus.cc:2781] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.328655 17884 raft_consensus.cc:686] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.329526 17884 consensus_queue.cc:237] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.333734 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:26.335924 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:26.336581 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:26.336483 17143 raft_consensus.cc:348] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.338310 17143 raft_consensus.cc:374] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.338560 17143 raft_consensus.cc:729] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.339898 17143 consensus_queue.cc:260] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.340459 17146 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:26.340924 17146 ts_tablet_manager.cc:1397] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.140s user 0.006s sys 0.004s
I20260923 17:31:26.343312 17146 raft_consensus.cc:348] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.343988 17146 raft_consensus.cc:374] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.344990 17144 raft_consensus.cc:348] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.346591 17144 raft_consensus.cc:374] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.346817 17144 raft_consensus.cc:729] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.347393 17144 consensus_queue.cc:260] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.348934 17146 raft_consensus.cc:729] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.349318 17144 ts_tablet_manager.cc:1428] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260923 17:31:26.340005 16385 catalog_manager.cc:5495] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:26.350123 17144 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.350386 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:26.352253 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:26.354290 17146 consensus_queue.cc:260] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.361129 17146 ts_tablet_manager.cc:1428] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260923 17:31:26.364720 17143 ts_tablet_manager.cc:1428] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.053s user 0.008s sys 0.000s
I20260923 17:31:26.365545 17143 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.372483 17144 tablet_bootstrap.cc:654] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.390480 17143 tablet_bootstrap.cc:654] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.394366 17834 raft_consensus.cc:482] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.394886 17834 raft_consensus.cc:504] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.402760 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.404132 16677 leader_election.cc:341] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.405813 17834 leader_election.cc:288] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.406910 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:26.407501 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:26.407996 17144 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.408396 17144 ts_tablet_manager.cc:1397] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.008s
I20260923 17:31:26.408341 16383 catalog_manager.cc:5495] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:26.410248 16383 catalog_manager.cc:5495] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:26.411897 16383 catalog_manager.cc:5495] T 373ad39e4d134ffc80caa73a83b80795 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:26.413326 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:26.411897 17144 raft_consensus.cc:348] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.414183 16383 catalog_manager.cc:5495] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:26.416347 17143 tablet_bootstrap.cc:492] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.416781 17143 ts_tablet_manager.cc:1397] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
I20260923 17:31:26.417486 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:26.418020 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:26.418960 17143 raft_consensus.cc:348] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.419565 17143 raft_consensus.cc:374] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.419800 17143 raft_consensus.cc:729] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.420360 17143 consensus_queue.cc:260] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.422154 17143 ts_tablet_manager.cc:1428] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:26.422577 17144 raft_consensus.cc:374] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.422868 17144 raft_consensus.cc:729] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.424888 17144 consensus_queue.cc:260] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.429447 17834 raft_consensus.cc:482] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.429868 17834 raft_consensus.cc:504] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.431562 17834 leader_election.cc:288] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.435871 17144 ts_tablet_manager.cc:1428] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260923 17:31:26.436673 17144 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.438234 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.439235 17033 raft_consensus.cc:3037] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.440982 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.441906 17035 raft_consensus.cc:2454] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:26.443499 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897f08f065fb4bc4bfa03853e4f696bc" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.444301 17037 raft_consensus.cc:3037] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.445683 17143 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.451867 17923 raft_consensus.cc:482] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.452298 17923 raft_consensus.cc:504] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.454030 17923 leader_election.cc:288] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.455513 17923 consensus_queue.cc:1035] T e329dd46a3a245b6826b01ac8de56621 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:26.456480 17144 tablet_bootstrap.cc:654] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.464464 17143 tablet_bootstrap.cc:654] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.465860 17859 raft_consensus.cc:482] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.466413 17146 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.466267 17859 raft_consensus.cc:504] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.469048 17144 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.469412 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.469410 17144 ts_tablet_manager.cc:1397] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.000s
I20260923 17:31:26.476694 17144 raft_consensus.cc:348] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.477411 17144 raft_consensus.cc:374] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.477931 17144 raft_consensus.cc:729] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.480894 17146 tablet_bootstrap.cc:654] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.481922 17144 consensus_queue.cc:260] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.484285 17144 ts_tablet_manager.cc:1428] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260923 17:31:26.486227 17144 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.486860 17808 raft_consensus.cc:482] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.490260 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92cfe501505458ea46da095a8c0d30a" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.468195 17859 leader_election.cc:288] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.492789 17815 raft_consensus.cc:482] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.493251 17815 raft_consensus.cc:504] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.470577 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:26.470822 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.495312 16677 leader_election.cc:341] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.495504 17815 leader_election.cc:288] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:26.495925 16677 leader_election.cc:341] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.498410 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.499251 17144 tablet_bootstrap.cc:654] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.487454 17808 raft_consensus.cc:504] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.499965 17035 raft_consensus.cc:2454] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:26.500876 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.501155 17808 leader_election.cc:288] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:26.501515 17039 raft_consensus.cc:2454] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:26.503199 16827 leader_election.cc:302] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:26.504557 17859 raft_consensus.cc:2726] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.510540 17867 raft_consensus.cc:482] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.504902 16678 leader_election.cc:302] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.511065 17867 raft_consensus.cc:504] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.512382 17834 raft_consensus.cc:2781] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.512714 17834 raft_consensus.cc:482] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.513005 17834 raft_consensus.cc:3037] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.513115 17867 leader_election.cc:288] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:26.515128 17859 raft_consensus.cc:482] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.515594 17859 raft_consensus.cc:504] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.518404 17144 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.518517 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.533733 17859 leader_election.cc:288] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.518808 17144 ts_tablet_manager.cc:1397] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.006s
W20260923 17:31:26.538036 16373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 -> Peer 012948ec54c1441381b43989690264ce (127.15.246.188:46075): Couldn't send request to peer 012948ec54c1441381b43989690264ce. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:46075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260923 17:31:26.539705 17144 raft_consensus.cc:348] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.540477 17144 raft_consensus.cc:374] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.540732 17144 raft_consensus.cc:729] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:26.547605 16677 consensus_peers.cc:487] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:26.548133 17144 consensus_queue.cc:260] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.548667 16678 leader_election.cc:302] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.558740 17032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6894a4ee5fe4884b5162d8055e766fe" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.559892 17843 raft_consensus.cc:2781] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:26.559795 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 65559us
I20260923 17:31:26.560261 17843 raft_consensus.cc:482] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.560549 17843 raft_consensus.cc:3037] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.561815 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.564071 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.564688 16739 raft_consensus.cc:2454] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:26.565677 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:26.567505 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.567781 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:26.568035 16739 raft_consensus.cc:2454] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:26.570454 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:26.570804 17144 ts_tablet_manager.cc:1428] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260923 17:31:26.574460 17146 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:26.574985 17146 ts_tablet_manager.cc:1397] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.003s
I20260923 17:31:26.574584 17843 raft_consensus.cc:504] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.575845 16827 leader_election.cc:302] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:26.576869 17859 raft_consensus.cc:2726] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.577589 17146 raft_consensus.cc:348] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.578260 17146 raft_consensus.cc:374] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.578539 17146 raft_consensus.cc:729] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.580667 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:26.582053 16677 leader_election.cc:341] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.577898 16678 leader_election.cc:302] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.585136 17843 leader_election.cc:288] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.587625 17033 raft_consensus.cc:2454] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.589291 17143 tablet_bootstrap.cc:492] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.589690 17143 ts_tablet_manager.cc:1397] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.144s user 0.017s sys 0.011s
I20260923 17:31:26.592402 17143 raft_consensus.cc:348] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.593158 17143 raft_consensus.cc:374] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.593397 17143 raft_consensus.cc:729] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.593956 17143 consensus_queue.cc:260] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.595851 17143 ts_tablet_manager.cc:1428] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260923 17:31:26.598893 17143 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.585742 17884 raft_consensus.cc:2781] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.586410 17144 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.600248 17884 raft_consensus.cc:482] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.600580 17884 raft_consensus.cc:3037] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.604882 17144 tablet_bootstrap.cc:654] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.606990 17884 raft_consensus.cc:504] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.617066 17143 tablet_bootstrap.cc:654] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.619583 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:26.620762 16677 leader_election.cc:341] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.627668 17144 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.628093 17144 ts_tablet_manager.cc:1397] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.003s
I20260923 17:31:26.631076 17143 tablet_bootstrap.cc:492] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.631470 17143 ts_tablet_manager.cc:1397] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.002s
W20260923 17:31:26.588514 16972 leader_election.cc:341] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.556047 17037 raft_consensus.cc:2454] T 897f08f065fb4bc4bfa03853e4f696bc P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.601600 17146 consensus_queue.cc:260] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:26.634210 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 45728.3us
W20260923 17:31:26.634960 16972 leader_election.cc:341] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:26.635797 16972 leader_election.cc:341] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.636298 17146 ts_tablet_manager.cc:1428] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.061s user 0.008s sys 0.000s
W20260923 17:31:26.636376 16972 leader_election.cc:341] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.636726 16972 leader_election.cc:302] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.638001 17808 raft_consensus.cc:2726] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.638688 17146 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.638912 17884 leader_election.cc:288] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.639863 16972 leader_election.cc:302] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.641074 17143 raft_consensus.cc:348] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.641441 17951 raft_consensus.cc:2781] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.641733 17143 raft_consensus.cc:374] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.641741 17951 raft_consensus.cc:482] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.641979 17143 raft_consensus.cc:729] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.642020 17951 raft_consensus.cc:3037] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.642514 17143 consensus_queue.cc:260] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.644438 17143 ts_tablet_manager.cc:1428] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20260923 17:31:26.648887 16972 leader_election.cc:302] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.649495 17143 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.649873 17952 raft_consensus.cc:2781] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.650175 17952 raft_consensus.cc:482] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.650453 17952 raft_consensus.cc:3037] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.661417 17143 tablet_bootstrap.cc:654] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.664855 17144 raft_consensus.cc:348] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.665576 17144 raft_consensus.cc:374] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.665838 17144 raft_consensus.cc:729] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.666390 17144 consensus_queue.cc:260] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.670272 17859 raft_consensus.cc:482] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.670593 17146 tablet_bootstrap.cc:654] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.670765 17859 raft_consensus.cc:504] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.671545 17952 raft_consensus.cc:504] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.673454 17884 raft_consensus.cc:482] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.673810 17952 leader_election.cc:288] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:26.673852 17884 raft_consensus.cc:504] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.675468 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:26.677585 17884 leader_election.cc:288] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.677742 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:26.678313 16739 raft_consensus.cc:3037] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.686851 16739 raft_consensus.cc:2454] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:26.687705 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.689321 16677 leader_election.cc:341] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:26.690168 16972 leader_election.cc:341] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.693197 16972 leader_election.cc:302] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.670094 17951 raft_consensus.cc:504] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.695448 17951 leader_election.cc:288] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:26.695600 16888 raft_consensus.cc:3037] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.699559 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:26.700964 17952 raft_consensus.cc:2781] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.701601 17952 raft_consensus.cc:686] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:26.704051 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:26.704788 16739 raft_consensus.cc:3037] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.705567 16678 leader_election.cc:302] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.705808 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.706871 17859 leader_election.cc:288] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.592074 17834 raft_consensus.cc:504] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.711027 17952 consensus_queue.cc:237] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:26.709095 16972 leader_election.cc:341] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.712283 17834 leader_election.cc:288] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.713029 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:26.714223 17923 raft_consensus.cc:2781] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.714722 17923 raft_consensus.cc:686] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.715076 17146 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:26.715497 17146 ts_tablet_manager.cc:1397] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.003s
I20260923 17:31:26.715809 17923 consensus_queue.cc:237] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.718482 17146 raft_consensus.cc:348] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:26.714280 16677 leader_election.cc:341] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.719146 17146 raft_consensus.cc:374] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.719386 17146 raft_consensus.cc:729] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.725108 16678 leader_election.cc:302] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.726598 17843 raft_consensus.cc:2781] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.727010 17843 raft_consensus.cc:686] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.727705 17843 consensus_queue.cc:237] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.728081 17146 consensus_queue.cc:260] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.731202 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2a0d1f4344ad69eb2c0a4e208839a" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.732967 16827 leader_election.cc:302] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:26.734643 17859 raft_consensus.cc:2726] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.752106 16739 raft_consensus.cc:2454] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 1.
I20260923 17:31:26.757121 16972 leader_election.cc:302] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.758970 17146 ts_tablet_manager.cc:1428] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.043s user 0.003s sys 0.003s
I20260923 17:31:26.758463 17952 raft_consensus.cc:2781] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.764461 17952 raft_consensus.cc:686] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Becoming Leader. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Running, Role: LEADER
I20260923 17:31:26.765544 17952 consensus_queue.cc:237] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.771816 17146 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.774201 17951 raft_consensus.cc:482] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.774696 17951 raft_consensus.cc:504] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.776683 17951 leader_election.cc:288] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:26.778209 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:26.780591 16972 leader_election.cc:341] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.782027 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:26.782568 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.783290 17033 raft_consensus.cc:3037] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:26.783257 16972 leader_election.cc:341] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:26.790833 16972 leader_election.cc:302] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:26.792574 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a2e5bc4ae186404c896814b95d6cc7ef consensus state has no leader
I20260923 17:31:26.798064 17923 consensus_queue.cc:1035] T 7efa62885c564a9bb1ca1b9beacf6e73 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:26.803045 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:26.803350 16888 raft_consensus.cc:3037] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.803853 17041 raft_consensus.cc:2454] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:26.805014 17031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.805575 17031 raft_consensus.cc:3037] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.806655 17808 raft_consensus.cc:2726] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.815152 17144 ts_tablet_manager.cc:1428] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.187s user 0.008s sys 0.029s
I20260923 17:31:26.818265 17144 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.822582 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:26.823163 17037 raft_consensus.cc:3037] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.828576 17146 tablet_bootstrap.cc:654] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.839000 17143 tablet_bootstrap.cc:492] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.839454 17143 ts_tablet_manager.cc:1397] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.190s user 0.016s sys 0.021s
I20260923 17:31:26.841820 17143 raft_consensus.cc:348] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.843509 17143 raft_consensus.cc:374] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.843793 17143 raft_consensus.cc:729] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.845120 17143 consensus_queue.cc:260] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.846225 17144 tablet_bootstrap.cc:654] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.851986 17143 ts_tablet_manager.cc:1428] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260923 17:31:26.854831 17031 raft_consensus.cc:2454] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.865343 16678 leader_election.cc:302] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.844067 17033 raft_consensus.cc:2454] T 1be2b507dfd54f4eb821ea2911b4508f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.873250 17143 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.873921 17964 raft_consensus.cc:2781] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:26.874291 17964 raft_consensus.cc:482] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:26.874665 17964 raft_consensus.cc:3037] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:26.878163 17143 tablet_bootstrap.cc:654] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.887749 17143 tablet_bootstrap.cc:492] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.888149 17143 ts_tablet_manager.cc:1397] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.001s
I20260923 17:31:26.890228 17143 raft_consensus.cc:348] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.890803 17143 raft_consensus.cc:374] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.891043 17143 raft_consensus.cc:729] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.894078 17146 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:26.894526 17146 ts_tablet_manager.cc:1397] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.123s user 0.013s sys 0.000s
I20260923 17:31:26.897322 17146 raft_consensus.cc:348] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:26.897974 17146 raft_consensus.cc:374] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.898203 17146 raft_consensus.cc:729] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.898955 17146 consensus_queue.cc:260] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:26.900776 17143 consensus_queue.cc:260] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.904148 17146 ts_tablet_manager.cc:1428] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260923 17:31:26.906184 17143 ts_tablet_manager.cc:1428] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260923 17:31:26.910434 17037 raft_consensus.cc:2454] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:26.913476 17144 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:26.913910 17144 ts_tablet_manager.cc:1397] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.096s user 0.010s sys 0.011s
I20260923 17:31:26.914208 17146 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:26.917994 17144 raft_consensus.cc:348] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.918587 17144 raft_consensus.cc:374] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.918812 17144 raft_consensus.cc:729] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.919307 17144 consensus_queue.cc:260] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.920178 17143 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.921696 17797 raft_consensus.cc:482] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.922202 17797 raft_consensus.cc:504] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:26.926348 16826 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:26.924046 17797 leader_election.cc:288] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:26.929576 16678 leader_election.cc:302] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
W20260923 17:31:26.929760 16827 leader_election.cc:341] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:26.930224 16827 leader_election.cc:302] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:26.931380 17797 raft_consensus.cc:2726] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:26.928194 17146 tablet_bootstrap.cc:654] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.935096 17143 tablet_bootstrap.cc:654] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.941501 17964 raft_consensus.cc:504] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.950515 17143 tablet_bootstrap.cc:492] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:26.950905 17143 ts_tablet_manager.cc:1397] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.003s
I20260923 17:31:26.950960 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 3d96d172fcb94589ae9cc36248176c17 (DEFAULT_TABLE table=default.Table_14b0bd687f5346b1b56e3bc95c7437ba [id=1f696cef697e4ada96f041624d84cfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.953125 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 3d96d172fcb94589ae9cc36248176c17 (DEFAULT_TABLE table=default.Table_14b0bd687f5346b1b56e3bc95c7437ba [id=1f696cef697e4ada96f041624d84cfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.954308 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d96d172fcb94589ae9cc36248176c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.954878 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:26.955715 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:26.944355 17923 raft_consensus.cc:2781] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.958096 17923 raft_consensus.cc:686] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.958127 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 3d96d172fcb94589ae9cc36248176c17 (DEFAULT_TABLE table=default.Table_14b0bd687f5346b1b56e3bc95c7437ba [id=1f696cef697e4ada96f041624d84cfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:26.959237 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d96d172fcb94589ae9cc36248176c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.958855 17923 consensus_queue.cc:237] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.945868 17144 ts_tablet_manager.cc:1428] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.032s user 0.005s sys 0.001s
I20260923 17:31:26.953153 17143 raft_consensus.cc:348] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.962347 17144 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:26.962468 17143 raft_consensus.cc:374] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:26.962710 17143 raft_consensus.cc:729] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:26.963340 17143 consensus_queue.cc:260] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.965287 17143 ts_tablet_manager.cc:1428] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260923 17:31:26.968739 17143 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:26.952195 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d96d172fcb94589ae9cc36248176c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:26.969215 17144 tablet_bootstrap.cc:654] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.957446 16678 leader_election.cc:302] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:26.972148 17966 raft_consensus.cc:2781] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:26.972585 17966 raft_consensus.cc:686] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:26.973305 17966 consensus_queue.cc:237] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.982401 17143 tablet_bootstrap.cc:654] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:26.993211 17952 raft_consensus.cc:482] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:26.993709 17952 raft_consensus.cc:504] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:26.995901 17952 leader_election.cc:288] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:26.997318 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:26.999477 16972 leader_election.cc:341] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.000316 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:27.001686 16972 leader_election.cc:341] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.002066 16972 leader_election.cc:302] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.003095 17952 raft_consensus.cc:2726] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.011657 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:27.013042 16677 leader_election.cc:341] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.013743 16678 leader_election.cc:302] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.023002 16870 tablet_service.cc:1467] Processing CreateTablet for tablet e17107ce0b3c4a739b60a2667a50963f (DEFAULT_TABLE table=default.Table_f833f76b0fe74b038732663e9c002a53 [id=289d455f4d6b4292bfe8f5e6ba6e8842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:27.025171 17146 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:27.025619 17146 ts_tablet_manager.cc:1397] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.112s user 0.007s sys 0.003s
I20260923 17:31:27.026432 16726 tablet_service.cc:1467] Processing CreateTablet for tablet e17107ce0b3c4a739b60a2667a50963f (DEFAULT_TABLE table=default.Table_f833f76b0fe74b038732663e9c002a53 [id=289d455f4d6b4292bfe8f5e6ba6e8842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:27.027411 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17107ce0b3c4a739b60a2667a50963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:27.031147 17146 raft_consensus.cc:348] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.031948 17146 raft_consensus.cc:374] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.032328 17146 raft_consensus.cc:729] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.032945 17146 consensus_queue.cc:260] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.023818 17843 raft_consensus.cc:2781] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:27.035867 17843 raft_consensus.cc:686] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:27.044049 17146 ts_tablet_manager.cc:1428] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.018s user 0.003s sys 0.004s
I20260923 17:31:27.044363 18003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.188:46075 every 8 connection(s)
I20260923 17:31:27.044395 17012 tablet_service.cc:1467] Processing CreateTablet for tablet e17107ce0b3c4a739b60a2667a50963f (DEFAULT_TABLE table=default.Table_f833f76b0fe74b038732663e9c002a53 [id=289d455f4d6b4292bfe8f5e6ba6e8842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:27.045598 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17107ce0b3c4a739b60a2667a50963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:27.058630 17244 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.188:46075
I20260923 17:31:27.063413 17244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
I20260923 17:31:27.069962 17146 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:27.074406 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 17244
I20260923 17:31:27.076365 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17107ce0b3c4a739b60a2667a50963f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:27.091858 17146 tablet_bootstrap.cc:654] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.029726 16388 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:27.110602 16380 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_632f010f9d114ecbab7901aa664556ac [id=17f994d2e127410685fffe47828a215b] alter complete (version 1)
I20260923 17:31:27.113349 17858 raft_consensus.cc:482] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.113922 17858 raft_consensus.cc:504] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.115060 16387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_6ff2f38012514a27ba6260d78aab79eb [id=6265c59de8be492aa44b9eb0f0f10c95] alter complete (version 1)
I20260923 17:31:27.116058 17858 leader_election.cc:288] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.118522 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.119282 17037 raft_consensus.cc:2379] T 6eb728f7f83e4975823e338a4568f796 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:27.156596 16387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_7a8e6e0a311345c1bb25f336834dc7f4 [id=843ad2ac09aa4360a5f46f738f20358d] alter complete (version 1)
I20260923 17:31:27.158089 16382 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 213) took 3180 ms. Trace:
I20260923 17:31:27.158604 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09c716342a6445a8edab130fd78a8e3" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:27.159238 17037 raft_consensus.cc:3037] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.162410 17146 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:27.162845 17146 ts_tablet_manager.cc:1397] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.093s user 0.008s sys 0.001s
I20260923 17:31:27.158366 16382 rpcz_store.cc:276] 0923 17:31:23.978048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.046990 (+ 68942us) service_pool.cc:227] Handling call
0923 17:31:24.047887 (+ 897us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:24.052213 (+ 4326us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:24.052835 (+ 622us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:24.053043 (+ 208us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:24.182840 (+129797us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:27.157923 (+2975083us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:24.128010 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:24.165847 (+ 37837us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:24.165900 (+ 53us) write_op.cc:435] Acquired schema lock
0923 17:31:24.165940 (+ 40us) tablet.cc:660] Decoding operations
0923 17:31:24.166491 (+ 551us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:24.166561 (+ 70us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:24.166612 (+ 51us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:24.166894 (+ 282us) tablet.cc:699] Row locks acquired
0923 17:31:24.166911 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:24.167147 (+ 236us) write_op.cc:270] Start()
0923 17:31:24.167299 (+ 152us) write_op.cc:276] Timestamp: P: 1790184684167060 usec, L: 0
0923 17:31:24.167347 (+ 48us) op_driver.cc:331] REPLICATION: starting
0923 17:31:24.168334 (+ 987us) log.cc:842] Serialized 405 byte log entry
0923 17:31:24.175335 (+ 7001us) op_driver.cc:444] REPLICATION: finished
0923 17:31:24.180101 (+ 4766us) write_op.cc:301] APPLY: starting
0923 17:31:24.180160 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:24.180269 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:24.180363 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:24.180511 (+ 148us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:24.180647 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:24.180686 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:24.181712 (+ 1026us) log.cc:842] Serialized 23 byte log entry
0923 17:31:24.182540 (+ 828us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:24.182609 (+ 69us) write_op.cc:454] Released schema lock
0923 17:31:24.182640 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:24.183029 (+ 389us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:24.183053 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4606,"apply.run_cpu_time_us":1497,"apply.run_wall_time_us":3183,"num_ops":1,"prepare.queue_time_us":69911,"prepare.run_cpu_time_us":2830,"prepare.run_wall_time_us":45916,"raft.queue_time_us":123,"raft.run_cpu_time_us":640,"raft.run_wall_time_us":640,"replication_time_us":7897}]]}
W20260923 17:31:27.165387 16382 rpcz_store.cc:267] Call kudu.master.MasterService.Ping from 127.15.246.130:46845 (request call id 35) took 1865 ms (client timeout 1000 ms). Trace:
I20260923 17:31:27.165094 17146 raft_consensus.cc:348] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.165645 17952 raft_consensus.cc:482] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.165715 17146 raft_consensus.cc:374] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260923 17:31:27.165796 16382 rpcz_store.cc:269] 0923 17:31:25.300145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.165106 (+1864961us) service_pool.cc:214] Skipping call since client already timed out
0923 17:31:27.165337 (+ 231us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260923 17:31:27.166129 17146 raft_consensus.cc:729] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.166177 17952 raft_consensus.cc:504] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:27.167161 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 40) took 1301 ms (client timeout 1000 ms). Trace:
I20260923 17:31:27.166867 17146 consensus_queue.cc:260] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.168161 17952 leader_election.cc:288] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:27.168882 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:27.169394 17146 ts_tablet_manager.cc:1428] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260923 17:31:27.170678 17037 raft_consensus.cc:2454] T e09c716342a6445a8edab130fd78a8e3 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
W20260923 17:31:27.171295 16972 leader_election.cc:341] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:27.167472 16382 rpcz_store.cc:269] 0923 17:31:25.865877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.167022 (+1301145us) service_pool.cc:214] Skipping call since client already timed out
0923 17:31:27.167105 (+ 83us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260923 17:31:27.173017 17146 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:27.182327 18006 sys_catalog.cc:263] Verifying existing consensus state
I20260923 17:31:27.187721 17146 tablet_bootstrap.cc:654] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.157275 17964 leader_election.cc:288] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.192001 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eb728f7f83e4975823e338a4568f796" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.036728 17843 consensus_queue.cc:237] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.205179 17144 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:27.205638 17144 ts_tablet_manager.cc:1397] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.243s user 0.014s sys 0.013s
I20260923 17:31:27.208704 17144 raft_consensus.cc:348] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.208604 18006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Bootstrap starting.
I20260923 17:31:27.209610 17144 raft_consensus.cc:374] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.209854 17144 raft_consensus.cc:729] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.210645 17144 consensus_queue.cc:260] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.213292 17144 ts_tablet_manager.cc:1428] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260923 17:31:27.216594 17144 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:27.222066 17146 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:27.222500 17146 ts_tablet_manager.cc:1397] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.000s
I20260923 17:31:27.224764 17146 raft_consensus.cc:348] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.225417 17146 raft_consensus.cc:374] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.225644 17146 raft_consensus.cc:729] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.226311 17146 consensus_queue.cc:260] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.228317 17146 ts_tablet_manager.cc:1428] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 17:31:27.229172 17146 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:27.235370 17146 tablet_bootstrap.cc:654] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.235998 17144 tablet_bootstrap.cc:654] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.246426 18006 log.cc:824] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:27.258109 17144 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:27.258560 17144 ts_tablet_manager.cc:1397] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.002s
I20260923 17:31:27.261035 17144 raft_consensus.cc:348] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.261709 17144 raft_consensus.cc:374] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.261963 17144 raft_consensus.cc:729] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.262507 17144 consensus_queue.cc:260] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.266160 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet df53a9825e5348189485f30b8fdedc63
I20260923 17:31:27.267619 16381 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 204) took 4905 ms. Trace:
I20260923 17:31:27.269492 17144 ts_tablet_manager.cc:1428] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.011s user 0.002s sys 0.005s
I20260923 17:31:27.270373 17144 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:27.272264 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:27.267894 16381 rpcz_store.cc:276] 0923 17:31:22.362444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.367021 (+1004577us) service_pool.cc:227] Handling call
0923 17:31:23.409094 (+ 42073us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:23.411975 (+ 2881us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:23.412093 (+ 118us) catalog_manager.cc:2761] Locking tablets
0923 17:31:23.421121 (+ 9028us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:23.475530 (+ 54409us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5400011680. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f91279a4361 00007f9127a74a86 00007f9127a6ae03 00007f9127a2fce7 00007f9127a2faf4 00007f91296101f1 00007f9129610a16 00007f91294c6052 00007f91294c2038 00007f91294c1883 00007f91295df49f
0923 17:31:23.725233 (+249703us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:23.725431 (+ 198us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:27.265938 (+3540507us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:27.267550 (+ 1612us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:23.575850 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:23.577888 (+ 2038us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:23.577919 (+ 31us) write_op.cc:435] Acquired schema lock
0923 17:31:23.577937 (+ 18us) tablet.cc:660] Decoding operations
0923 17:31:23.578339 (+ 402us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:23.578388 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:23.578422 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:23.578742 (+ 320us) tablet.cc:699] Row locks acquired
0923 17:31:23.578763 (+ 21us) write_op.cc:260] PREPARE: finished
0923 17:31:23.578887 (+ 124us) write_op.cc:270] Start()
0923 17:31:23.578974 (+ 87us) write_op.cc:276] Timestamp: P: 1790184683578837 usec, L: 0
0923 17:31:23.578991 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:23.579525 (+ 534us) log.cc:842] Serialized 768 byte log entry
0923 17:31:23.721079 (+141554us) op_driver.cc:444] REPLICATION: finished
0923 17:31:23.721488 (+ 409us) write_op.cc:301] APPLY: starting
0923 17:31:23.721547 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:23.721671 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:23.721767 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:23.721996 (+ 229us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:23.722145 (+ 149us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:23.722187 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:23.722379 (+ 192us) log.cc:842] Serialized 29 byte log entry
0923 17:31:23.722521 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.722589 (+ 68us) write_op.cc:454] Released schema lock
0923 17:31:23.722628 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:23.722791 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:23.722814 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":119095168,"child_traces":[["op",{"apply.queue_time_us":253,"apply.run_cpu_time_us":1504,"apply.run_wall_time_us":1504,"num_ops":2,"prepare.queue_time_us":100152,"prepare.run_cpu_time_us":2370,"prepare.run_wall_time_us":5067,"raft.queue_time_us":1101,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":142005}]]}
W20260923 17:31:27.277509 16381 rpcz_store.cc:267] Call kudu.master.MasterService.Ping from 127.15.246.130:46845 (request call id 36) took 972 ms (client timeout 1000 ms). Trace:
W20260923 17:31:27.277879 16381 rpcz_store.cc:269] 0923 17:31:26.304779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.277336 (+972557us) service_pool.cc:227] Handling call
0923 17:31:27.277463 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:27.279131 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.285197 17144 tablet_bootstrap.cc:654] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.288359 18020 tablet_replica.cc:330] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:27.289522 16386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_dcf1d5122d0c421bab95f4f6e66d3f47 [id=6d073df311674a358a520dd1586a3345] alter complete (version 3)
I20260923 17:31:27.290478 18020 raft_consensus.cc:2229] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:27.291090 18020 raft_consensus.cc:2258] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:27.291874 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.292963 16940 heartbeater.cc:347] Connected to a master server at 127.15.246.190:33707
I20260923 17:31:27.294317 16888 raft_consensus.cc:3037] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.296870 17146 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:27.297324 17146 ts_tablet_manager.cc:1397] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.068s user 0.006s sys 0.005s
I20260923 17:31:27.301564 18020 ts_tablet_manager.cc:1905] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:27.304925 17146 raft_consensus.cc:348] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.309332 17146 raft_consensus.cc:374] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.309643 17146 raft_consensus.cc:729] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.310432 17146 consensus_queue.cc:260] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.311244 17144 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:27.311642 17144 ts_tablet_manager.cc:1397] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.001s
I20260923 17:31:27.315217 17144 raft_consensus.cc:348] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.320662 17144 raft_consensus.cc:374] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.320978 17144 raft_consensus.cc:729] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.321579 17144 consensus_queue.cc:260] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.329695 17146 ts_tablet_manager.cc:1428] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260923 17:31:27.330406 17859 raft_consensus.cc:482] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.330914 17859 raft_consensus.cc:504] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.331218 17146 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:27.332729 17859 leader_election.cc:288] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.333923 17144 ts_tablet_manager.cc:1428] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260923 17:31:27.334436 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.334847 17144 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:27.335211 17037 raft_consensus.cc:2379] T 1822edfa26b045d7a15074a4bd5cbf5e P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:27.338054 17146 tablet_bootstrap.cc:654] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.339993 17144 tablet_bootstrap.cc:654] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.300650 16379 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_c191488ea4084de1a6d09e6c6e93b1ed [id=ee6fa9253d554437bb19b8eae678f66c] alter complete (version 1)
I20260923 17:31:27.348693 18024 tablet_replica.cc:330] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:27.349557 18024 raft_consensus.cc:2229] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:27.360961 18024 raft_consensus.cc:2258] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:27.362131 17951 raft_consensus.cc:482] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.362700 17951 raft_consensus.cc:504] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.370451 17951 leader_election.cc:288] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:27.372210 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c115e29673747f8b0712d441ac993fb" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:27.373826 16972 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.375372 17143 tablet_bootstrap.cc:492] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:27.375931 17143 ts_tablet_manager.cc:1397] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.407s user 0.042s sys 0.049s
I20260923 17:31:27.379619 17144 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:27.380100 17144 ts_tablet_manager.cc:1397] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.005s
I20260923 17:31:27.386662 17144 raft_consensus.cc:348] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.387398 17144 raft_consensus.cc:374] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.387653 17144 raft_consensus.cc:729] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.392797 18021 tablet_replica.cc:330] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:27.396865 17144 consensus_queue.cc:260] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.398768 17144 ts_tablet_manager.cc:1428] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260923 17:31:27.402650 16678 leader_election.cc:302] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.403644 17753 raft_consensus.cc:2781] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:27.404294 17753 raft_consensus.cc:686] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:27.405411 17753 consensus_queue.cc:237] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.409600 17143 raft_consensus.cc:348] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.410943 18021 raft_consensus.cc:2229] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:27.411038 17144 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:27.415500 17859 raft_consensus.cc:482] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.416031 17859 raft_consensus.cc:504] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.418622 17859 leader_election.cc:288] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.419711 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.420547 17041 raft_consensus.cc:2379] T 47a0f16066164132a26927f5a830e677 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:27.424516 16826 leader_election.cc:302] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:27.425768 17859 raft_consensus.cc:3037] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.433082 17859 raft_consensus.cc:2726] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.437465 18024 ts_tablet_manager.cc:1905] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:27.410284 17143 raft_consensus.cc:374] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.446307 17143 raft_consensus.cc:729] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.446213 17834 raft_consensus.cc:482] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.446695 17834 raft_consensus.cc:504] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.446900 17143 consensus_queue.cc:260] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.449187 17834 leader_election.cc:288] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.453073 17884 raft_consensus.cc:482] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.453486 17884 raft_consensus.cc:504] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.455395 17884 leader_election.cc:288] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.458070 18013 consensus_queue.cc:1035] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:27.464783 16891 raft_consensus.cc:3037] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.489820 18013 raft_consensus.cc:482] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.490341 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.490360 18013 raft_consensus.cc:504] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.492069 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c115e29673747f8b0712d441ac993fb" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.492964 18013 leader_election.cc:288] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:27.493758 16972 leader_election.cc:341] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.494151 16972 leader_election.cc:302] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.495153 17951 raft_consensus.cc:2726] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:27.497619 16972 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.498009 16972 leader_election.cc:302] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.499197 17952 raft_consensus.cc:2726] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.499850 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1822edfa26b045d7a15074a4bd5cbf5e" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.500850 17143 ts_tablet_manager.cc:1428] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.124s user 0.006s sys 0.000s
I20260923 17:31:27.501348 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a0f16066164132a26927f5a830e677" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.501752 17143 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:27.503427 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.504079 17033 raft_consensus.cc:2454] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:27.506450 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:27.507393 16826 leader_election.cc:302] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:27.508184 17797 raft_consensus.cc:3037] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.508451 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.510962 16826 leader_election.cc:302] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:27.511574 17041 raft_consensus.cc:2454] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:27.511698 17858 raft_consensus.cc:3037] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.523962 17884 consensus_queue.cc:1035] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:27.521106 17753 consensus_queue.cc:1035] T 6eb728f7f83e4975823e338a4568f796 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:27.528695 16677 leader_election.cc:341] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.529196 16886 raft_consensus.cc:3037] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.533255 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:27.536687 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:27.539709 16893 raft_consensus.cc:3037] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:27.554663 18006 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.066s user 0.002s sys 0.000s
I20260923 17:31:27.565361 16938 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:27.566265 17146 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:27.566709 17146 ts_tablet_manager.cc:1397] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.236s user 0.010s sys 0.000s
W20260923 17:31:27.568529 16677 leader_election.cc:341] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.391492 16387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_11aad91009934d2fa94936fa3f7b98ba [id=2e8bb6c24f454c97882afd519641822f] alter complete (version 2)
W20260923 17:31:27.569110 16677 leader_election.cc:341] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.579087 17858 raft_consensus.cc:2726] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.586604 17144 tablet_bootstrap.cc:654] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.590342 17086 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:27.590706 17146 raft_consensus.cc:348] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.591354 17146 raft_consensus.cc:374] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.591580 17146 raft_consensus.cc:729] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.592269 17146 consensus_queue.cc:260] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.595996 17797 raft_consensus.cc:2726] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.597980 17753 consensus_queue.cc:1035] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:27.579432 17143 tablet_bootstrap.cc:654] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.620814 18040 tablet.cc:1722] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:27.621950 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:27.628882 17951 raft_consensus.cc:482] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.629441 17951 raft_consensus.cc:504] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:27.632100 17951 leader_election.cc:288] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:27.635826 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:27.639834 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.639959 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.640115 17753 consensus_queue.cc:1035] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:27.641105 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.641734 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.642871 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.643474 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2beb8360d54efeb2fa74c005d011d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:13 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.644150 18024 ts_tablet_manager.cc:1918] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:27.646847 18024 log.cc:1196] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/df53a9825e5348189485f30b8fdedc63
I20260923 17:31:27.647303 17753 consensus_queue.cc:1035] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:27.648824 16678 leader_election.cc:302] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.649428 17753 raft_consensus.cc:2781] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:27.649750 17753 raft_consensus.cc:482] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:27.650048 17753 raft_consensus.cc:3037] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.653131 18024 ts_tablet_manager.cc:1939] T df53a9825e5348189485f30b8fdedc63 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:27.660750 18021 raft_consensus.cc:2258] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:27.662977 18021 ts_tablet_manager.cc:1905] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:27.667236 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 33) took 4732 ms (client timeout 1000 ms). Trace:
I20260923 17:31:27.667446 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c842eff86634bf4bc70c426b5226d66 with delete_type TABLET_DATA_DELETED (replaced by tablet 3208c90d96f349988acb6705f2f438d5) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.670006 18043 tablet.cc:1722] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:27.672624 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:27.674013 16972 leader_election.cc:341] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.674364 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:27.674930 17041 raft_consensus.cc:2454] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:27.676172 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:27.676774 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.677882 18024 tablet_replica.cc:330] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:27.678342 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet a86f4b7ee14948f0989af54b3b77dc71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.680814 18024 raft_consensus.cc:2229] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:27.681313 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.683369 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.682029 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.682713 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:27.680210 16972 consensus_peers.cc:487] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:27.684468 16972 leader_election.cc:341] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.684788 16972 leader_election.cc:302] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.682821 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 with delete_type TABLET_DATA_DELETED (replaced by tablet b7b4d39a636e44e68ef2229006551ffa) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:27.685704 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.685838 17952 raft_consensus.cc:2726] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:27.670539 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet df53a9825e5348189485f30b8fdedc63 (table default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]) successfully deleted
W20260923 17:31:27.676947 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 34) took 3447 ms (client timeout 1000 ms). Trace:
W20260923 17:31:27.689024 16386 rpcz_store.cc:269] 0923 17:31:24.180906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.229030 (+ 48124us) service_pool.cc:227] Handling call
0923 17:31:27.628202 (+3399172us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:27.271311 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:27.271711 (+ 400us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:27.271741 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:27.271751 (+ 10us) tablet.cc:660] Decoding operations
0923 17:31:27.272013 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:27.272056 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:27.272084 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:27.272244 (+ 160us) tablet.cc:699] Row locks acquired
0923 17:31:27.272261 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:27.272375 (+ 114us) write_op.cc:270] Start()
0923 17:31:27.272472 (+ 97us) write_op.cc:276] Timestamp: P: 1790184687272349 usec, L: 0
0923 17:31:27.272486 (+ 14us) op_driver.cc:331] REPLICATION: starting
0923 17:31:27.272959 (+ 473us) log.cc:842] Serialized 356 byte log entry
0923 17:31:27.282290 (+ 9331us) op_driver.cc:444] REPLICATION: finished
0923 17:31:27.282666 (+ 376us) write_op.cc:301] APPLY: starting
0923 17:31:27.282716 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:27.282811 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:27.282915 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:27.283116 (+ 201us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:27.283233 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:27.283280 (+ 47us) write_op.cc:312] APPLY: finished
0923 17:31:27.283463 (+ 183us) log.cc:842] Serialized 23 byte log entry
0923 17:31:27.283596 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.283656 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:27.283686 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:27.283850 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.283875 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":70224,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":1360,"apply.run_wall_time_us":1358,"num_ops":1,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":1936,"prepare.run_wall_time_us":1937,"raft.queue_time_us":2284,"raft.run_cpu_time_us":626,"raft.run_wall_time_us":626,"replication_time_us":9726}]]}
I20260923 17:31:27.691478 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.685045 17884 raft_consensus.cc:482] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:27.697391 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ccd82f191942239ffb88459d362107 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a2f1143dcd64b2dbd25e17787096903) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.697966 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.698840 17884 raft_consensus.cc:504] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.702872 17884 leader_election.cc:288] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.694077 17753 raft_consensus.cc:504] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.729626 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:27.732753 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:27.732091 17753 leader_election.cc:288] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:27.734293 17143 tablet_bootstrap.cc:492] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:27.734814 17143 ts_tablet_manager.cc:1397] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.233s user 0.025s sys 0.002s
I20260923 17:31:27.736630 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:27.738686 16677 leader_election.cc:341] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:27.704568 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 2c842eff86634bf4bc70c426b5226d66 because the tablet was not found. No further retry: Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:27.743484 16720 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.744369 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet a80034cbf6354e1bbe0a0220e43251c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c115e29673747f8b0712d441ac993fb) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.745141 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet e818f6192de6472cb27747c0e3a1f128 with delete_type TABLET_DATA_DELETED (replaced by tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:27.746744 16677 consensus_peers.cc:487] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:27.711874 16387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 29) took 4853 ms (client timeout 1000 ms). Trace:
I20260923 17:31:27.749809 18024 raft_consensus.cc:2258] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:27.751880 18024 ts_tablet_manager.cc:1905] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:27.755831 17143 raft_consensus.cc:348] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.756767 17143 raft_consensus.cc:374] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:27.757081 17143 raft_consensus.cc:729] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:27.757921 17143 consensus_queue.cc:260] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.761382 17143 ts_tablet_manager.cc:1428] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.026s user 0.006s sys 0.001s
W20260923 17:31:27.685832 16380 rpcz_store.cc:269] 0923 17:31:22.934878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.987186 (+ 52308us) service_pool.cc:227] Handling call
0923 17:31:27.667120 (+4679934us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:24.445519 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:24.445968 (+ 449us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:24.445995 (+ 27us) write_op.cc:435] Acquired schema lock
0923 17:31:24.446011 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:24.446318 (+ 307us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:24.446364 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:24.446389 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:24.446553 (+ 164us) tablet.cc:699] Row locks acquired
0923 17:31:24.446570 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:24.446696 (+ 126us) write_op.cc:270] Start()
0923 17:31:24.446801 (+ 105us) write_op.cc:276] Timestamp: P: 1790184684446648 usec, L: 0
0923 17:31:24.446816 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:24.447397 (+ 581us) log.cc:842] Serialized 356 byte log entry
0923 17:31:24.513019 (+ 65622us) op_driver.cc:444] REPLICATION: finished
0923 17:31:24.515125 (+ 2106us) write_op.cc:301] APPLY: starting
0923 17:31:24.515177 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:24.515265 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:24.515347 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:24.515501 (+ 154us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:24.515630 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:24.515669 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:24.515858 (+ 189us) log.cc:842] Serialized 23 byte log entry
0923 17:31:24.515993 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:24.516044 (+ 51us) write_op.cc:454] Released schema lock
0923 17:31:24.516073 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:24.516255 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:24.516280 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":84706,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":1969,"apply.run_cpu_time_us":1299,"apply.run_wall_time_us":1299,"num_ops":1,"prepare.queue_time_us":107355,"prepare.run_cpu_time_us":2282,"prepare.run_wall_time_us":67589,"raft.queue_time_us":3227,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":295,"replication_time_us":66123,"spinlock_wait_cycles":127744}]]}
I20260923 17:31:27.772342 18021 ts_tablet_manager.cc:1918] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:27.772850 17143 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:27.772799 18021 log.cc:1196] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/df53a9825e5348189485f30b8fdedc63
I20260923 17:31:27.773810 18021 ts_tablet_manager.cc:1939] T df53a9825e5348189485f30b8fdedc63 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:27.776346 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet df53a9825e5348189485f30b8fdedc63 (table default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]) successfully deleted
W20260923 17:31:27.777700 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a60bf118082b4e9991bfeb3ac59eb118 because the tablet was not found. No further retry: Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
W20260923 17:31:27.778877 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet c8895cffac5945a19a5de7fba1319f22 because the tablet was not found. No further retry: Not found: Tablet not found: c8895cffac5945a19a5de7fba1319f22
W20260923 17:31:27.780046 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet f05a6f83314d433a9502e0c00ba8ad09 because the tablet was not found. No further retry: Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
W20260923 17:31:27.781754 16972 consensus_peers.cc:487] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:27.783077 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 2c842eff86634bf4bc70c426b5226d66 because the tablet was not found. No further retry: Not found: Tablet not found: 2c842eff86634bf4bc70c426b5226d66
I20260923 17:31:27.783871 18021 tablet_replica.cc:330] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:27.766739 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:27.788285 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 36) took 5256 ms (client timeout 1000 ms). Trace:
W20260923 17:31:27.790270 16677 leader_election.cc:341] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.801385 17143 tablet_bootstrap.cc:654] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:27.815474 17951 consensus_queue.cc:1035] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:27.816988 17144 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:27.817939 17144 ts_tablet_manager.cc:1397] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.408s user 0.010s sys 0.003s
W20260923 17:31:27.698081 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 49342.3us
I20260923 17:31:27.824661 16678 leader_election.cc:302] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.825433 17884 raft_consensus.cc:2781] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:27.825778 17884 raft_consensus.cc:482] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:27.826299 17884 raft_consensus.cc:3037] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.840201 16678 leader_election.cc:302] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:27.862571 17146 ts_tablet_manager.cc:1428] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.296s user 0.005s sys 0.000s
I20260923 17:31:27.865248 17146 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:27.869179 16384 rpcz_store.cc:269] 0923 17:31:22.531662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.726610 (+194948us) service_pool.cc:227] Handling call
0923 17:31:27.787441 (+5060831us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22074,"reported_tablets":30}
I20260923 17:31:27.872007 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_37d4870a361e4247a261b101b89d1ffb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:27.881294 18006 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.059s user 0.001s sys 0.000s
W20260923 17:31:27.892438 18043 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.221s user 0.001s sys 0.000s
I20260923 17:31:27.880787 18020 ts_tablet_manager.cc:1918] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:27.889117 18060 tablet.cc:1722] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:27.909341 18020 log.cc:1196] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/df53a9825e5348189485f30b8fdedc63
I20260923 17:31:27.911505 18020 ts_tablet_manager.cc:1939] T df53a9825e5348189485f30b8fdedc63 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:27.867202 18040 tablet.cc:1722] T e92cfe501505458ea46da095a8c0d30a P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:27.874645 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.876184 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet c8895cffac5945a19a5de7fba1319f22 with delete_type TABLET_DATA_DELETED (replaced by tablet b77ef164608f4ced8b1c2ecab6f88c1a) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.877804 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0c75099d4d43339e88bf7527b10af9 with delete_type TABLET_DATA_DELETED (replaced by tablet b7b4d39a636e44e68ef2229006551ffa) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:27.835688 17884 raft_consensus.cc:504] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.923166 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:27.925323 16677 leader_election.cc:341] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.918805 18064 tablet.cc:1722] T e92cfe501505458ea46da095a8c0d30a P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:27.931201 18040 tablet.cc:1722] T a6894a4ee5fe4884b5162d8055e766fe P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:27.934078 18012 raft_consensus.cc:2781] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:27.934394 18012 raft_consensus.cc:482] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:27.934684 18012 raft_consensus.cc:3037] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:27.944098 17884 leader_election.cc:288] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:27.944927 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 104819us
I20260923 17:31:27.931833 17921 consensus_queue.cc:1035] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:27.943915 18012 raft_consensus.cc:504] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:27.954282 18024 ts_tablet_manager.cc:1918] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:27.954878 18024 log.cc:1196] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:27.955790 18024 ts_tablet_manager.cc:1939] T a60bf118082b4e9991bfeb3ac59eb118 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:27.957290 18012 leader_election.cc:288] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:27.734669 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 38) took 3964 ms (client timeout 1000 ms). Trace:
W20260923 17:31:27.792681 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 30) took 3832 ms (client timeout 1000 ms). Trace:
W20260923 17:31:27.964406 16385 rpcz_store.cc:269] 0923 17:31:23.960114 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.991643 (+ 31529us) service_pool.cc:227] Handling call
0923 17:31:26.377873 (+2386230us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.377896 (+ 23us) write_op.cc:454] Released schema lock
0923 17:31:27.792600 (+1414704us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:26.359878 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:26.360876 (+ 998us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:26.360900 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:26.360913 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:26.361185 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:26.361228 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:26.361251 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:26.361402 (+ 151us) tablet.cc:699] Row locks acquired
0923 17:31:26.361415 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:26.361528 (+ 113us) write_op.cc:270] Start()
0923 17:31:26.361638 (+ 110us) write_op.cc:276] Timestamp: P: 1790184686361478 usec, L: 0
0923 17:31:26.361653 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:26.362191 (+ 538us) log.cc:842] Serialized 536 byte log entry
0923 17:31:26.372023 (+ 9832us) op_driver.cc:444] REPLICATION: finished
0923 17:31:26.372347 (+ 324us) write_op.cc:301] APPLY: starting
0923 17:31:26.372403 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:26.372494 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:26.372600 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:26.372750 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:26.372880 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:26.372920 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:26.373091 (+ 171us) log.cc:842] Serialized 23 byte log entry
0923 17:31:26.373244 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.373306 (+ 62us) write_op.cc:454] Released schema lock
0923 17:31:26.373334 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":110611,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":185,"apply.run_cpu_time_us":1139,"apply.run_wall_time_us":1138,"num_ops":1,"prepare.queue_time_us":177,"prepare.run_cpu_time_us":2594,"prepare.run_wall_time_us":2594,"raft.queue_time_us":945,"raft.run_cpu_time_us":675,"raft.run_wall_time_us":674,"replication_time_us":10297}]]}
I20260923 17:31:27.970600 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_6df3e9d4c07446dd9e203188dcb3329c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:27.957233 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:27.980649 16677 leader_election.cc:341] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:27.974239 17143 tablet_bootstrap.cc:492] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:27.982472 17143 ts_tablet_manager.cc:1397] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.210s user 0.011s sys 0.012s
W20260923 17:31:27.988914 18006 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.105s user 0.003s sys 0.000s
I20260923 17:31:27.989753 18006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 17:31:27.991269 18006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Bootstrap complete.
W20260923 17:31:27.737269 16383 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 32) took 2710 ms (client timeout 1000 ms). Trace:
I20260923 17:31:28.001111 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet a60bf118082b4e9991bfeb3ac59eb118 (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
I20260923 17:31:27.985090 17143 raft_consensus.cc:348] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.001641 17143 raft_consensus.cc:374] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.002252 17143 raft_consensus.cc:729] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:28.002360 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a86f4b7ee14948f0989af54b3b77dc71 because the tablet was not found. No further retry: Not found: Tablet not found: a86f4b7ee14948f0989af54b3b77dc71
I20260923 17:31:28.002840 17143 consensus_queue.cc:260] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.004403 18024 tablet_replica.cc:330] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:27.997690 16383 rpcz_store.cc:269] 0923 17:31:25.026580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:25.098197 (+ 71617us) service_pool.cc:227] Handling call
0923 17:31:26.583890 (+1485693us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.583916 (+ 26us) write_op.cc:454] Released schema lock
0923 17:31:27.737200 (+1153284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:26.421186 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:26.421611 (+ 425us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:26.421642 (+ 31us) write_op.cc:435] Acquired schema lock
0923 17:31:26.421662 (+ 20us) tablet.cc:660] Decoding operations
0923 17:31:26.422257 (+ 595us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:26.422299 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:26.422325 (+ 26us) tablet.cc:683] Acquiring locks for 4 operations
0923 17:31:26.422679 (+ 354us) tablet.cc:699] Row locks acquired
0923 17:31:26.422693 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:26.422813 (+ 120us) write_op.cc:270] Start()
0923 17:31:26.422926 (+ 113us) write_op.cc:276] Timestamp: P: 1790184686422760 usec, L: 0
0923 17:31:26.422947 (+ 21us) op_driver.cc:331] REPLICATION: starting
0923 17:31:26.423430 (+ 483us) log.cc:842] Serialized 1686 byte log entry
0923 17:31:26.473544 (+ 50114us) op_driver.cc:444] REPLICATION: finished
0923 17:31:26.473818 (+ 274us) write_op.cc:301] APPLY: starting
0923 17:31:26.473880 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:26.474015 (+ 135us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:26.474098 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:26.474494 (+ 396us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:26.474627 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0923 17:31:26.474667 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:26.474909 (+ 242us) log.cc:842] Serialized 41 byte log entry
0923 17:31:26.475046 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.475117 (+ 71us) write_op.cc:454] Released schema lock
0923 17:31:26.475148 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":43461,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":141,"apply.run_cpu_time_us":1492,"apply.run_wall_time_us":1492,"num_ops":4,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":2497,"prepare.run_wall_time_us":2498,"raft.queue_time_us":668,"raft.run_cpu_time_us":515,"raft.run_wall_time_us":515,"replication_time_us":50529}]]}
I20260923 17:31:28.008668 16383 master_service.cc:946] ReplaceTablet: received request to replace tablet 19ebfb3cfd3d49b39c7cf453199ac8bd from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:28.014501 17143 ts_tablet_manager.cc:1428] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.032s user 0.002s sys 0.003s
W20260923 17:31:28.014425 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6df3e9d4c07446dd9e203188dcb3329c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:28.014700 18024 raft_consensus.cc:2229] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:28.015971 18024 raft_consensus.cc:2258] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:27.927078 17952 consensus_queue.cc:1035] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:27.973501 18068 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.018735 18024 ts_tablet_manager.cc:1905] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.021752 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet df53a9825e5348189485f30b8fdedc63 (table default.Table_3542e5d4b3e34da19c70ab50d8a95080 [id=ecd32ef6694843a7a239931ce715f405]) successfully deleted
W20260923 17:31:28.023223 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet c8895cffac5945a19a5de7fba1319f22 because the tablet was not found. No further retry: Not found: Tablet not found: c8895cffac5945a19a5de7fba1319f22
W20260923 17:31:28.023778 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet f05a6f83314d433a9502e0c00ba8ad09 because the tablet was not found. No further retry: Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
W20260923 17:31:28.024330 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 1e0c75099d4d43339e88bf7527b10af9 because the tablet was not found. No further retry: Not found: Tablet not found: 1e0c75099d4d43339e88bf7527b10af9
I20260923 17:31:28.026542 18021 raft_consensus.cc:2229] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:27.972071 18064 tablet.cc:1722] T a6894a4ee5fe4884b5162d8055e766fe P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:28.031942 18013 consensus_queue.cc:1035] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:28.033097 17921 raft_consensus.cc:482] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.034667 18021 raft_consensus.cc:2258] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:28.036764 18021 ts_tablet_manager.cc:1905] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.033514 17921 raft_consensus.cc:504] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.037583 17143 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.038687 17921 leader_election.cc:288] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.034641 18013 raft_consensus.cc:482] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.043100 18013 raft_consensus.cc:504] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.046723 18013 leader_election.cc:288] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:28.049176 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 98037us
W20260923 17:31:27.748569 16387 rpcz_store.cc:269] 0923 17:31:22.858969 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:22.969938 (+110969us) service_pool.cc:227] Handling call
0923 17:31:25.024354 (+2054416us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:25.024387 (+ 33us) write_op.cc:454] Released schema lock
0923 17:31:25.123974 (+ 99587us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:25.124001 (+ 27us) write_op.cc:454] Released schema lock
0923 17:31:27.145138 (+2021137us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.145161 (+ 23us) write_op.cc:454] Released schema lock
0923 17:31:27.711798 (+566637us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:24.964131 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:24.964607 (+ 476us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:24.964633 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:24.964645 (+ 12us) tablet.cc:660] Decoding operations
0923 17:31:24.965000 (+ 355us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:24.965040 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:24.965064 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:24.965364 (+ 300us) tablet.cc:699] Row locks acquired
0923 17:31:24.965377 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:24.965499 (+ 122us) write_op.cc:270] Start()
0923 17:31:24.965590 (+ 91us) write_op.cc:276] Timestamp: P: 1790184684965451 usec, L: 0
0923 17:31:24.965605 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:24.966921 (+ 1316us) log.cc:842] Serialized 919 byte log entry
0923 17:31:24.976632 (+ 9711us) op_driver.cc:444] REPLICATION: finished
0923 17:31:24.977408 (+ 776us) write_op.cc:301] APPLY: starting
0923 17:31:24.977475 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:24.977602 (+ 127us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:24.977702 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:24.977939 (+ 237us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:24.978089 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:24.978133 (+ 44us) write_op.cc:312] APPLY: finished
0923 17:31:24.978309 (+ 176us) log.cc:842] Serialized 29 byte log entry
0923 17:31:24.978446 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:24.978515 (+ 69us) write_op.cc:454] Released schema lock
0923 17:31:24.978550 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0923 17:31:25.100582 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:25.101212 (+ 630us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:25.101246 (+ 34us) write_op.cc:435] Acquired schema lock
0923 17:31:25.101261 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:25.101598 (+ 337us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:25.101647 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:25.101678 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:25.101924 (+ 246us) tablet.cc:699] Row locks acquired
0923 17:31:25.101937 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:25.102169 (+ 232us) write_op.cc:270] Start()
0923 17:31:25.102274 (+ 105us) write_op.cc:276] Timestamp: P: 1790184685102107 usec, L: 0
0923 17:31:25.102290 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:25.102940 (+ 650us) log.cc:842] Serialized 356 byte log entry
0923 17:31:25.114770 (+ 11830us) op_driver.cc:444] REPLICATION: finished
0923 17:31:25.115171 (+ 401us) write_op.cc:301] APPLY: starting
0923 17:31:25.115236 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:25.115333 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:25.115452 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:25.115653 (+ 201us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:25.115788 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:25.115840 (+ 52us) write_op.cc:312] APPLY: finished
0923 17:31:25.116046 (+ 206us) log.cc:842] Serialized 23 byte log entry
0923 17:31:25.116205 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:25.116296 (+ 91us) write_op.cc:454] Released schema lock
0923 17:31:25.116334 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0923 17:31:27.120760 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:27.121170 (+ 410us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:27.121195 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:27.121209 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:27.121486 (+ 277us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:27.121526 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:27.121553 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:27.121690 (+ 137us) tablet.cc:699] Row locks acquired
0923 17:31:27.121705 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:27.121823 (+ 118us) write_op.cc:270] Start()
0923 17:31:27.121901 (+ 78us) write_op.cc:276] Timestamp: P: 1790184687121760 usec, L: 0
0923 17:31:27.121918 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:27.122436 (+ 518us) log.cc:842] Serialized 356 byte log entry
0923 17:31:27.141988 (+ 19552us) op_driver.cc:444] REPLICATION: finished
0923 17:31:27.142315 (+ 327us) write_op.cc:301] APPLY: starting
0923 17:31:27.142377 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:27.142458 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:27.142547 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:27.142720 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:27.142852 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:27.142893 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:27.143053 (+ 160us) log.cc:842] Serialized 23 byte log entry
0923 17:31:27.143177 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.143237 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:27.143265 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0923 17:31:27.345650 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:27.346096 (+ 446us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:27.346121 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:27.346136 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:27.346410 (+ 274us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:27.346447 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:27.346471 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:27.346604 (+ 133us) tablet.cc:699] Row locks acquired
0923 17:31:27.346618 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:27.346741 (+ 123us) write_op.cc:270] Start()
0923 17:31:27.346831 (+ 90us) write_op.cc:276] Timestamp: P: 1790184687346715 usec, L: 0
0923 17:31:27.346851 (+ 20us) op_driver.cc:331] REPLICATION: starting
0923 17:31:27.355150 (+ 8299us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b5400012398. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f91283370c4 00007f91275e34f9 00007f912759ef91 00007f912759ec6a 00007f912760acd5 00007f912760b179 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9
0923 17:31:27.355509 (+ 359us) log.cc:842] Serialized 356 byte log entry
0923 17:31:27.382328 (+ 26819us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b58000a04b0. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127589834 00007f912758e771 00007f912758e6f9 00007f912758e689 00007f912758e651 00007f912758d81d 0000000000580784 00000000005805b9 00007f91206ba6b6 00007f91206bdb11
0923 17:31:27.382571 (+ 243us) op_driver.cc:444] REPLICATION: finished
0923 17:31:27.382831 (+ 260us) write_op.cc:301] APPLY: starting
0923 17:31:27.382876 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:27.383009 (+ 133us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:27.383092 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:27.383348 (+ 256us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:27.383474 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:27.383511 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:27.383670 (+ 159us) log.cc:842] Serialized 23 byte log entry
0923 17:31:27.383828 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.383890 (+ 62us) write_op.cc:454] Released schema lock
0923 17:31:27.383920 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:27.384081 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:27.384103 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":177340,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":610,"apply.run_cpu_time_us":1316,"apply.run_wall_time_us":1316,"num_ops":2,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":2456,"prepare.run_wall_time_us":23079,"raft.queue_time_us":100,"raft.run_cpu_time_us":626,"raft.run_wall_time_us":627,"replication_time_us":10948}],["op",{"apply.queue_time_us":225,"apply.run_cpu_time_us":1374,"apply.run_wall_time_us":1412,"num_ops":1,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":2611,"prepare.run_wall_time_us":23331,"raft.queue_time_us":95,"raft.run_cpu_time_us":738,"raft.run_wall_time_us":737,"replication_time_us":12388}],["op",{"apply.queue_time_us":201,"apply.run_cpu_time_us":1127,"apply.run_wall_time_us":1127,"num_ops":1,"prepare.queue_time_us":290,"prepare.run_cpu_time_us":2012,"prepare.run_wall_time_us":2013,"raft.queue_time_us":2632,"raft.run_cpu_time_us":713,"raft.run_wall_time_us":714,"replication_time_us":20005}],["op",{"apply.queue_time_us":140,"apply.run_cpu_time_us":1413,"apply.run_wall_time_us":1412,"num_ops":1,"prepare.queue_time_us":24230,"prepare.run_cpu_time_us":3379,"prepare.run_wall_time_us":21314,"raft.queue_time_us":245,"raft.run_cpu_time_us":2348,"raft.run_wall_time_us":26664,"replication_time_us":35664,"spinlock_wait_cycles":70833920}]]}
I20260923 17:31:28.049474 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:28.040350 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.065824 16677 leader_election.cc:341] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:27.960877 16379 rpcz_store.cc:269] 0923 17:31:23.770238 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:23.770374 (+ 136us) service_pool.cc:227] Handling call
0923 17:31:26.327383 (+2557009us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.327411 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:27.734587 (+1407176us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:26.082311 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:26.082790 (+ 479us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:26.082820 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:26.082837 (+ 17us) tablet.cc:660] Decoding operations
0923 17:31:26.083237 (+ 400us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:26.083282 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:26.083305 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:26.083610 (+ 305us) tablet.cc:699] Row locks acquired
0923 17:31:26.083626 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:26.083758 (+ 132us) write_op.cc:270] Start()
0923 17:31:26.083859 (+ 101us) write_op.cc:276] Timestamp: P: 1790184686083692 usec, L: 0
0923 17:31:26.083877 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:26.084493 (+ 616us) log.cc:842] Serialized 919 byte log entry
0923 17:31:26.100881 (+ 16388us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b1800058480. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f91295008b1 00007f912950d661 00007f91206ba638 00007f91206bdb11 00007f91206bdac9 00007f91206bda59 00007f91206bda21 00007f91206bcd1d 0000000000580784 00000000005805b9 00007f912069cdc6
0923 17:31:26.170180 (+ 69299us) op_driver.cc:444] REPLICATION: finished
0923 17:31:26.218188 (+ 48008us) write_op.cc:301] APPLY: starting
0923 17:31:26.218249 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:26.218360 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:26.218378 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:26.218701 (+ 323us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:26.218864 (+ 163us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:26.218904 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:26.219117 (+ 213us) log.cc:842] Serialized 29 byte log entry
0923 17:31:26.219275 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.219360 (+ 85us) write_op.cc:454] Released schema lock
0923 17:31:26.219397 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:26.280900 (+ 61503us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.280927 (+ 27us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:26.295642 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:26.296035 (+ 393us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:26.296061 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:26.296075 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:26.296389 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:26.296430 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:26.296453 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:26.296597 (+ 144us) tablet.cc:699] Row locks acquired
0923 17:31:26.296611 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:26.296721 (+ 110us) write_op.cc:270] Start()
0923 17:31:26.296800 (+ 79us) write_op.cc:276] Timestamp: P: 1790184686296680 usec, L: 0
0923 17:31:26.296816 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:26.297300 (+ 484us) log.cc:842] Serialized 356 byte log entry
0923 17:31:26.310838 (+ 13538us) op_driver.cc:444] REPLICATION: finished
0923 17:31:26.320844 (+ 10006us) write_op.cc:301] APPLY: starting
0923 17:31:26.320900 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:26.324192 (+ 3292us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:26.324344 (+ 152us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:26.324540 (+ 196us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:26.324665 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:26.324707 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:26.324881 (+ 174us) log.cc:842] Serialized 23 byte log entry
0923 17:31:26.325027 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:26.325090 (+ 63us) write_op.cc:454] Released schema lock
0923 17:31:26.325118 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":42082,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":47796,"apply.run_cpu_time_us":1804,"apply.run_wall_time_us":62972,"num_ops":2,"prepare.queue_time_us":73505,"prepare.run_cpu_time_us":2586,"prepare.run_wall_time_us":2587,"raft.queue_time_us":572,"raft.run_cpu_time_us":774,"raft.run_wall_time_us":774,"replication_time_us":86218,"spinlock_wait_cycles":8037760,"thread_start_us":73724,"threads_started":1}],["op",{"apply.queue_time_us":9835,"apply.run_cpu_time_us":1292,"apply.run_wall_time_us":4457,"num_ops":1,"prepare.queue_time_us":268,"prepare.run_cpu_time_us":1961,"prepare.run_wall_time_us":1962,"raft.queue_time_us":912,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":606,"replication_time_us":13949}]]}
W20260923 17:31:28.066517 16677 leader_election.cc:341] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.068099 16379 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_0d464342027a447fbc67d549f98de051" } new_table_name: "default.Table_b0ce21ff6a6e46c8a4a37cdc3ab4f877" modify_external_catalogs: true
W20260923 17:31:27.974320 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37d4870a361e4247a261b101b89d1ffb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:27.878597 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 39) took 3055 ms (client timeout 1000 ms). Trace:
W20260923 17:31:28.069416 16388 rpcz_store.cc:269] 0923 17:31:24.823582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:24.876731 (+ 53149us) service_pool.cc:227] Handling call
0923 17:31:27.878514 (+3001783us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37688,"reported_tablets":30}
I20260923 17:31:28.071432 16388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_8be9935ea009465aaaae2c81f5fa6783" } modify_external_catalogs: true
I20260923 17:31:28.068445 17144 raft_consensus.cc:348] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.080443 17144 raft_consensus.cc:374] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.081468 17144 raft_consensus.cc:729] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.082762 17144 consensus_queue.cc:260] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.085103 18024 ts_tablet_manager.cc:1918] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:28.085492 16387 master_service.cc:946] ReplaceTablet: received request to replace tablet 7c115e29673747f8b0712d441ac993fb from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:28.085587 18024 log.cc:1196] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:28.087191 17143 tablet_bootstrap.cc:654] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.087149 18024 ts_tablet_manager.cc:1939] T f9f3ed1bd50d4e2eb30e1a6b6ee8178d P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:28.018286 18068 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.108786 18068 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:28.111001 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d (table default.Table_2243b815d62f461ca66f77ac0701c362 [id=86e881974c3f45d2a4f21cdb6cda848c]) successfully deleted
I20260923 17:31:28.111361 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.112852 16972 leader_election.cc:341] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:28.113023 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 42ccd82f191942239ffb88459d362107 because the tablet was not found. No further retry: Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
I20260923 17:31:28.113993 18024 tablet_replica.cc:330] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:28.091836 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.114522 17037 raft_consensus.cc:2454] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:28.101749 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.115178 18024 raft_consensus.cc:2229] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:28.115242 17033 raft_consensus.cc:3037] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.022887 18020 tablet_replica.cc:330] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:27.984192 17146 tablet_bootstrap.cc:654] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.116384 18024 raft_consensus.cc:2258] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 17:31:28.118062 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7dbd8eb0ab9243bd81c3d6a893f18aea consensus state has no leader
I20260923 17:31:28.122692 18024 ts_tablet_manager.cc:1905] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.132846 17033 raft_consensus.cc:2454] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.132642 18060 tablet.cc:1722] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:28.111281 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:28.151170 16972 leader_election.cc:341] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.151582 16972 leader_election.cc:302] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.152379 18068 raft_consensus.cc:2726] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.152837 17144 ts_tablet_manager.cc:1428] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.335s user 0.007s sys 0.000s
I20260923 17:31:28.164004 17952 raft_consensus.cc:482] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.164520 17952 raft_consensus.cc:504] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.166250 17952 leader_election.cc:288] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:28.166638 17952 raft_consensus.cc:482] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.166976 17952 raft_consensus.cc:504] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.170202 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:28.171530 16677 consensus_peers.cc:487] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:28.171480 16972 leader_election.cc:341] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.172555 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.174042 16972 leader_election.cc:341] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.174441 16972 leader_election.cc:302] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.175746 17952 leader_election.cc:288] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:28.177785 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:28.179153 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.180400 16972 leader_election.cc:341] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:28.181038 16972 leader_election.cc:341] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.181376 16972 leader_election.cc:302] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.182391 18059 raft_consensus.cc:2726] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.182914 17952 raft_consensus.cc:2726] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.185350 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.186014 17036 raft_consensus.cc:2454] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:28.187072 17031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.187695 17031 raft_consensus.cc:3037] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.195461 17035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.196173 17035 raft_consensus.cc:2454] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:28.185387 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31a83c7168a64b64bd224ef5a4c6939e" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.198619 17033 raft_consensus.cc:3037] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.198908 16388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_8be9935ea009465aaaae2c81f5fa6783" } modify_external_catalogs: true
I20260923 17:31:28.218173 17033 raft_consensus.cc:2454] T 31a83c7168a64b64bd224ef5a4c6939e P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.219424 18024 ts_tablet_manager.cc:1918] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:28.220970 17031 raft_consensus.cc:2454] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.220332 18037 raft_consensus.cc:482] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.226295 18037 raft_consensus.cc:504] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.226851 17143 tablet_bootstrap.cc:492] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:28.230528 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.231518 17033 raft_consensus.cc:2379] T c7e6178ab07245e3a687d53eb2ff03ab P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:28.228253 18037 leader_election.cc:288] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.232138 18024 log.cc:1196] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/051f2e478a324df98647d90107d55533
I20260923 17:31:28.232451 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e6178ab07245e3a687d53eb2ff03ab" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:28.232955 18024 ts_tablet_manager.cc:1939] T 051f2e478a324df98647d90107d55533 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:28.236922 17144 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:28.238193 16827 leader_election.cc:302] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:28.239046 16886 raft_consensus.cc:3037] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.239010 18037 raft_consensus.cc:3037] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.232271 17143 ts_tablet_manager.cc:1397] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.195s user 0.015s sys 0.018s
I20260923 17:31:28.241979 17143 raft_consensus.cc:348] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.242666 17143 raft_consensus.cc:374] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.242908 17143 raft_consensus.cc:729] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.245913 18037 raft_consensus.cc:2726] T c7e6178ab07245e3a687d53eb2ff03ab P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.248517 17143 consensus_queue.cc:260] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.249961 17144 tablet_bootstrap.cc:654] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.250491 17143 ts_tablet_manager.cc:1428] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260923 17:31:28.264837 18012 raft_consensus.cc:482] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.265292 18012 raft_consensus.cc:504] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.266414 17143 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.267244 18012 leader_election.cc:288] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.270107 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:28.271503 17143 tablet_bootstrap.cc:654] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:28.271668 16677 leader_election.cc:341] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.275489 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 051f2e478a324df98647d90107d55533 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
I20260923 17:31:28.275700 18024 tablet_replica.cc:330] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:28.276721 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet c8895cffac5945a19a5de7fba1319f22 because the tablet was not found. No further retry: Not found: Tablet not found: c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:28.279733 17144 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:28.280200 17144 ts_tablet_manager.cc:1397] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.000s
I20260923 17:31:28.282542 17144 raft_consensus.cc:348] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.283214 17144 raft_consensus.cc:374] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.291750 17143 tablet_bootstrap.cc:492] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:28.283442 17144 raft_consensus.cc:729] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.292155 17143 ts_tablet_manager.cc:1397] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.000s
I20260923 17:31:28.293511 17144 consensus_queue.cc:260] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.295444 17144 ts_tablet_manager.cc:1428] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.015s user 0.003s sys 0.002s
I20260923 17:31:28.295709 17143 raft_consensus.cc:348] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.312506 17146 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:28.296344 17143 raft_consensus.cc:374] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.336630 17143 raft_consensus.cc:729] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.337184 17143 consensus_queue.cc:260] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.338586 17143 ts_tablet_manager.cc:1428] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.045s user 0.003s sys 0.000s
I20260923 17:31:28.339378 17143 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.355232 17144 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:28.362308 17144 tablet_bootstrap.cc:654] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.318037 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.363543 17039 raft_consensus.cc:2454] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
W20260923 17:31:28.364697 18076 log.cc:924] Time spent T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Append to log took a long time: real 0.089s user 0.001s sys 0.000s
I20260923 17:31:28.365417 17146 ts_tablet_manager.cc:1397] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.500s user 0.010s sys 0.001s
I20260923 17:31:28.382872 17144 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:28.383328 17144 ts_tablet_manager.cc:1397] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.001s
I20260923 17:31:28.386919 18012 consensus_queue.cc:1035] T 1822edfa26b045d7a15074a4bd5cbf5e P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:28.389261 17144 raft_consensus.cc:348] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.389968 17144 raft_consensus.cc:374] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.390236 17144 raft_consensus.cc:729] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.390924 17144 consensus_queue.cc:260] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.391544 17923 consensus_queue.cc:1035] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:28.392876 18021 ts_tablet_manager.cc:1918] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:28.393396 16678 leader_election.cc:302] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.393484 18021 log.cc:1196] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/051f2e478a324df98647d90107d55533
I20260923 17:31:28.393013 17144 ts_tablet_manager.cc:1428] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20260923 17:31:28.394687 18021 ts_tablet_manager.cc:1939] T 051f2e478a324df98647d90107d55533 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:28.401180 18020 raft_consensus.cc:2229] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:28.404657 18020 raft_consensus.cc:2258] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:28.407150 18020 ts_tablet_manager.cc:1905] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.367835 17146 raft_consensus.cc:348] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.417964 17146 raft_consensus.cc:374] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.418579 17146 raft_consensus.cc:729] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.419275 17146 consensus_queue.cc:260] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.422730 17146 ts_tablet_manager.cc:1428] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.057s user 0.003s sys 0.002s
I20260923 17:31:28.439707 18024 raft_consensus.cc:2229] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:28.441186 18024 raft_consensus.cc:2258] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:28.444818 18024 ts_tablet_manager.cc:1905] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.451025 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:28.452752 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:28.455461 16383 catalog_manager.cc:6565] ReplaceTablet: tablet 19ebfb3cfd3d49b39c7cf453199ac8bd deleted and replaced by tablet d15e626ea47f435b8cc75d91d3d04900
I20260923 17:31:28.455884 16383 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 226) took 3655 ms. Trace:
I20260923 17:31:28.456239 16383 rpcz_store.cc:276] 0923 17:31:24.800895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.008463 (+3207568us) service_pool.cc:227] Handling call
0923 17:31:28.455830 (+447367us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.297260 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.297708 (+ 448us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.297734 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:28.297749 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:28.298075 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.298137 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.298160 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.368115 (+ 69955us) tablet.cc:699] Row locks acquired
0923 17:31:28.368130 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:28.368284 (+ 154us) write_op.cc:270] Start()
0923 17:31:28.368375 (+ 91us) write_op.cc:276] Timestamp: P: 1790184688368235 usec, L: 0
0923 17:31:28.368390 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.368931 (+ 541us) log.cc:842] Serialized 665 byte log entry
0923 17:31:28.423496 (+ 54565us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.445042 (+ 21546us) write_op.cc:301] APPLY: starting
0923 17:31:28.445101 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.445205 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.445302 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.445666 (+ 364us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.446041 (+ 375us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:28.446086 (+ 45us) write_op.cc:312] APPLY: finished
0923 17:31:28.446275 (+ 189us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.446420 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.446491 (+ 71us) write_op.cc:454] Released schema lock
0923 17:31:28.446520 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":21354,"num_ops":2,"prepare.queue_time_us":115598,"prepare.run_cpu_time_us":2308,"prepare.run_wall_time_us":71972,"raft.queue_time_us":35964,"raft.run_cpu_time_us":1374,"raft.run_wall_time_us":1375,"replication_time_us":55038,"thread_start_us":140196,"threads_started":2}]]}
I20260923 17:31:28.394922 17144 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:28.479355 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 051f2e478a324df98647d90107d55533 (table default.Table_b518db3a4e274e35a617e1c2b35f43e6 [id=fb7dea1e18e144ea99beb878d935f942]) successfully deleted
W20260923 17:31:28.482254 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet eb2beb8360d54efeb2fa74c005d011d1 because the tablet was not found. No further retry: Not found: Tablet not found: eb2beb8360d54efeb2fa74c005d011d1
I20260923 17:31:28.436468 17923 raft_consensus.cc:2781] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260923 17:31:28.484130 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 42ccd82f191942239ffb88459d362107 because the tablet was not found. No further retry: Not found: Tablet not found: 42ccd82f191942239ffb88459d362107
W20260923 17:31:28.437817 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 44510.8us
I20260923 17:31:28.454555 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:28.484424 17923 raft_consensus.cc:482] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:28.486784 17923 raft_consensus.cc:3037] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.497460 16678 leader_election.cc:302] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.500022 18012 raft_consensus.cc:2781] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:28.500635 18012 raft_consensus.cc:686] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:28.501468 18012 consensus_queue.cc:237] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.504429 17144 tablet_bootstrap.cc:654] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.515244 17753 raft_consensus.cc:482] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.515753 17753 raft_consensus.cc:504] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.518692 17753 leader_election.cc:288] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.520476 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:28.522456 17923 raft_consensus.cc:504] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.459851 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:28.525967 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because the tablet was not found. No further retry: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
W20260923 17:31:28.527025 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 547fda6e7bcf474e9f73ba9a6659d34f because the tablet was not found. No further retry: Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
W20260923 17:31:28.528040 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet a80034cbf6354e1bbe0a0220e43251c3 because the tablet was not found. No further retry: Not found: Tablet not found: a80034cbf6354e1bbe0a0220e43251c3
W20260923 17:31:28.528584 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet e818f6192de6472cb27747c0e3a1f128 because the tablet was not found. No further retry: Not found: Tablet not found: e818f6192de6472cb27747c0e3a1f128
W20260923 17:31:28.529481 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet c8895cffac5945a19a5de7fba1319f22 because the tablet was not found. No further retry: Not found: Tablet not found: c8895cffac5945a19a5de7fba1319f22
I20260923 17:31:28.535169 16384 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 224) took 3847 ms. Trace:
I20260923 17:31:28.505082 16678 leader_election.cc:302] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.535480 16384 rpcz_store.cc:276] 0923 17:31:24.688264 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.871398 (+3183134us) service_pool.cc:227] Handling call
0923 17:31:28.202078 (+330680us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:28.203367 (+ 1289us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:28.534448 (+331081us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:28.534855 (+ 407us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:28.535109 (+ 254us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.369330 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.369741 (+ 411us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.369765 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:28.369783 (+ 18us) tablet.cc:660] Decoding operations
0923 17:31:28.370069 (+ 286us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.370105 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.370139 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.370320 (+ 181us) tablet.cc:699] Row locks acquired
0923 17:31:28.370335 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:28.370432 (+ 97us) write_op.cc:270] Start()
0923 17:31:28.370509 (+ 77us) write_op.cc:276] Timestamp: P: 1790184688370391 usec, L: 0
0923 17:31:28.370527 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.371221 (+ 694us) log.cc:842] Serialized 433 byte log entry
0923 17:31:28.447030 (+ 75809us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.498090 (+ 51060us) write_op.cc:301] APPLY: starting
0923 17:31:28.498157 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.498314 (+ 157us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.498500 (+ 186us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.533313 (+ 34813us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.533452 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:28.533513 (+ 61us) write_op.cc:312] APPLY: finished
0923 17:31:28.533825 (+ 312us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.534143 (+ 318us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.534209 (+ 66us) write_op.cc:454] Released schema lock
0923 17:31:28.534246 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.534439 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.534461 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":50768,"apply.run_cpu_time_us":2427,"apply.run_wall_time_us":36631,"num_ops":2,"prepare.queue_time_us":164938,"prepare.run_cpu_time_us":2296,"prepare.run_wall_time_us":52188,"raft.queue_time_us":35229,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":76431,"thread_start_us":52083,"threads_started":1}]]}
W20260923 17:31:28.538205 16678 consensus_peers.cc:487] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:28.538962 18013 raft_consensus.cc:2781] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:28.539314 18013 raft_consensus.cc:482] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:28.539610 18013 raft_consensus.cc:3037] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.539561 16384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_96a86405cb7a4a9493b667daf383bf1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:28.540741 17923 leader_election.cc:288] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.541958 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:28.544754 16384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96a86405cb7a4a9493b667daf383bf1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:28.545707 16677 leader_election.cc:341] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:28.547785 16677 consensus_peers.cc:487] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:28.548667 17143 tablet_bootstrap.cc:654] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.549397 16938 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:28.549784 17144 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:28.550230 17144 ts_tablet_manager.cc:1397] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.156s user 0.009s sys 0.004s
I20260923 17:31:28.553292 18069 raft_consensus.cc:482] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.553869 17144 raft_consensus.cc:348] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.554512 17144 raft_consensus.cc:374] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.554715 17144 raft_consensus.cc:729] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
W20260923 17:31:28.556128 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1e0c75099d4d43339e88bf7527b10af9 because the tablet was not found. No further retry: Not found: Tablet not found: 1e0c75099d4d43339e88bf7527b10af9
W20260923 17:31:28.557440 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet df53a9825e5348189485f30b8fdedc63 because the tablet was not found. No further retry: Not found: Tablet not found: df53a9825e5348189485f30b8fdedc63
I20260923 17:31:28.557135 18013 raft_consensus.cc:504] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:28.560413 16677 leader_election.cc:341] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.553745 18069 raft_consensus.cc:504] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.562958 16678 leader_election.cc:302] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.563117 18069 leader_election.cc:288] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:28.563923 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:28.564811 17921 raft_consensus.cc:2781] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:28.565140 17921 raft_consensus.cc:482] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:28.565469 17921 raft_consensus.cc:3037] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.575642 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:28.576921 16972 leader_election.cc:341] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.577564 17923 consensus_queue.cc:1035] T 47a0f16066164132a26927f5a830e677 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:28.574882 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.579566 16972 leader_election.cc:341] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.579972 16972 leader_election.cc:302] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.580485 17146 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:28.582038 16677 leader_election.cc:341] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.582469 17143 tablet_bootstrap.cc:492] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:28.583029 17143 ts_tablet_manager.cc:1397] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.244s user 0.007s sys 0.007s
I20260923 17:31:28.584398 18068 raft_consensus.cc:2726] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.585659 17143 raft_consensus.cc:348] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.586333 17143 raft_consensus.cc:374] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.586573 17143 raft_consensus.cc:729] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.587288 17143 consensus_queue.cc:260] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.589397 17143 ts_tablet_manager.cc:1428] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260923 17:31:28.590858 17921 raft_consensus.cc:504] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.583946 16678 leader_election.cc:302] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.596140 17923 raft_consensus.cc:2781] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:28.599552 16678 leader_election.cc:302] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.597546 17923 raft_consensus.cc:686] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:28.603433 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:28.592082 17144 consensus_queue.cc:260] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.604583 17884 consensus_queue.cc:1035] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:28.605561 16677 leader_election.cc:341] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.605075 17923 consensus_queue.cc:237] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.608734 17144 ts_tablet_manager.cc:1428] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.058s user 0.008s sys 0.001s
I20260923 17:31:28.609671 17144 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:28.600471 18013 leader_election.cc:288] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.603248 17753 raft_consensus.cc:2781] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:28.611414 17753 raft_consensus.cc:686] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:28.612602 17753 consensus_queue.cc:237] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.620438 18012 raft_consensus.cc:482] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.621260 17143 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.620091 17921 leader_election.cc:288] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.621179 18012 raft_consensus.cc:504] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.625310 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:28.629181 16677 leader_election.cc:341] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.631425 17146 tablet_bootstrap.cc:654] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.642663 18013 consensus_queue.cc:1035] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:28.643680 17884 consensus_queue.cc:1035] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:28.632386 18012 leader_election.cc:288] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.663023 17144 tablet_bootstrap.cc:654] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.565025 18021 tablet_replica.cc:330] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:28.671276 17143 tablet_bootstrap.cc:654] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.673010 18021 raft_consensus.cc:2229] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:28.674845 18021 raft_consensus.cc:2258] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:28.677855 18021 ts_tablet_manager.cc:1905] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:28.684921 18037 raft_consensus.cc:482] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.685694 18037 raft_consensus.cc:504] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:28.702050 16827 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:28.703009 18037 leader_election.cc:288] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:28.708415 16826 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:28.708823 16826 leader_election.cc:302] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:28.715574 16884 raft_consensus.cc:3037] T 897f08f065fb4bc4bfa03853e4f696bc P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.716957 18037 raft_consensus.cc:2726] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.717911 18024 ts_tablet_manager.cc:1918] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:28.718372 18024 log.cc:1196] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:28.719125 18024 ts_tablet_manager.cc:1939] T f1ccdf1bd7a64f51965e0a6dc7e033c9 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:28.721480 18024 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 214) took 1038 ms. Trace:
I20260923 17:31:28.721863 18024 rpcz_store.cc:276] 0923 17:31:27.683468 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.685571 (+ 2103us) service_pool.cc:227] Handling call
0923 17:31:28.275560 (+589989us) ts_tablet_manager.cc:1145] Deleting tablet f1ccdf1bd7a64f51965e0a6dc7e033c9
0923 17:31:28.275598 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:28.666039 (+390441us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:28.717645 (+ 51606us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:28.721417 (+ 3772us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":162485,"tablet-delete.queue_time_us":584168}
I20260923 17:31:28.723486 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 (table default.Table_e69cd0d31b464f32a2c86b830f18b7df [id=4a760b075c324bdebcbdaea430cef553]) successfully deleted
I20260923 17:31:28.724669 18024 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 215) took 1038 ms. Trace:
I20260923 17:31:28.725044 18024 rpcz_store.cc:276] 0923 17:31:27.686409 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.689096 (+ 2687us) service_pool.cc:227] Handling call
0923 17:31:28.724370 (+1035274us) ts_tablet_manager.cc:1145] Deleting tablet e818f6192de6472cb27747c0e3a1f128
0923 17:31:28.724416 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:28.724614 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1032487}
W20260923 17:31:28.726440 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet e818f6192de6472cb27747c0e3a1f128 because the tablet was not found. No further retry: Not found: Tablet not found: e818f6192de6472cb27747c0e3a1f128
I20260923 17:31:28.727555 18024 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 216) took 1029 ms. Trace:
I20260923 17:31:28.727909 18024 rpcz_store.cc:276] 0923 17:31:27.698342 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.698447 (+ 105us) service_pool.cc:227] Handling call
0923 17:31:28.727248 (+1028801us) ts_tablet_manager.cc:1145] Deleting tablet a80034cbf6354e1bbe0a0220e43251c3
0923 17:31:28.727294 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:28.727504 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":997130}
W20260923 17:31:28.729156 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a80034cbf6354e1bbe0a0220e43251c3 because the tablet was not found. No further retry: Not found: Tablet not found: a80034cbf6354e1bbe0a0220e43251c3
I20260923 17:31:28.730007 18024 tablet_replica.cc:330] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:28.732467 17146 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:28.735529 17146 ts_tablet_manager.cc:1397] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.182s user 0.010s sys 0.004s
I20260923 17:31:28.746378 17146 raft_consensus.cc:348] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.760300 17146 raft_consensus.cc:374] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.761183 17146 raft_consensus.cc:729] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:28.761618 16379 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b77ef164608f4ced8b1c2ecab6f88c1a consensus state has no leader
I20260923 17:31:28.762151 16379 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 227) took 3922 ms. Trace:
I20260923 17:31:28.763159 17146 consensus_queue.cc:260] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.762517 16379 rpcz_store.cc:276] 0923 17:31:24.839980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.067694 (+3227714us) service_pool.cc:227] Handling call
0923 17:31:28.204506 (+136812us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:28.206108 (+ 1602us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:28.206647 (+ 539us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:28.206885 (+ 238us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:28.692454 (+485569us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:28.762099 (+ 69645us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.421768 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.422785 (+ 1017us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.422820 (+ 35us) write_op.cc:435] Acquired schema lock
0923 17:31:28.422836 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:28.423219 (+ 383us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.473277 (+ 50058us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.473503 (+ 226us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:28.473803 (+ 300us) tablet.cc:699] Row locks acquired
0923 17:31:28.473848 (+ 45us) write_op.cc:260] PREPARE: finished
0923 17:31:28.474101 (+ 253us) write_op.cc:270] Start()
0923 17:31:28.474307 (+ 206us) write_op.cc:276] Timestamp: P: 1790184688474017 usec, L: 0
0923 17:31:28.474357 (+ 50us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.509414 (+ 35057us) log.cc:842] Serialized 405 byte log entry
0923 17:31:28.545644 (+ 36230us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:28.685061 (+139417us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.688288 (+ 3227us) write_op.cc:301] APPLY: starting
0923 17:31:28.688346 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.688487 (+ 141us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.688506 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.688687 (+ 181us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.688805 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:28.688844 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:28.689148 (+ 304us) log.cc:842] Serialized 23 byte log entry
0923 17:31:28.689453 (+ 305us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.689534 (+ 81us) write_op.cc:454] Released schema lock
0923 17:31:28.689631 (+ 97us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.689995 (+ 364us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.690064 (+ 69us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3023,"apply.run_cpu_time_us":1989,"apply.run_wall_time_us":1989,"num_ops":1,"prepare.queue_time_us":213815,"prepare.run_cpu_time_us":7013,"prepare.run_wall_time_us":124075,"replication_time_us":210531,"spinlock_wait_cycles":76110720}]]}
I20260923 17:31:28.765974 16385 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 225) took 4000 ms. Trace:
I20260923 17:31:28.766944 17146 ts_tablet_manager.cc:1428] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.031s user 0.004s sys 0.000s
I20260923 17:31:28.767093 17143 tablet_bootstrap.cc:492] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:28.767681 17143 ts_tablet_manager.cc:1397] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.147s user 0.007s sys 0.021s
I20260923 17:31:28.766379 16385 rpcz_store.cc:276] 0923 17:31:24.765728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.969992 (+3204264us) service_pool.cc:227] Handling call
0923 17:31:28.196482 (+226490us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:28.197946 (+ 1464us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:28.695380 (+497434us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:28.765648 (+ 70268us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:28.765922 (+ 274us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.545898 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.546324 (+ 426us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.546393 (+ 69us) write_op.cc:435] Acquired schema lock
0923 17:31:28.546441 (+ 48us) tablet.cc:660] Decoding operations
0923 17:31:28.546841 (+ 400us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.546882 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.546913 (+ 31us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.547339 (+ 426us) tablet.cc:699] Row locks acquired
0923 17:31:28.547380 (+ 41us) write_op.cc:260] PREPARE: finished
0923 17:31:28.547635 (+ 255us) write_op.cc:270] Start()
0923 17:31:28.547728 (+ 93us) write_op.cc:276] Timestamp: P: 1790184688547552 usec, L: 0
0923 17:31:28.547748 (+ 20us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.566812 (+ 19064us) log.cc:842] Serialized 433 byte log entry
0923 17:31:28.685466 (+118654us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.690537 (+ 5071us) write_op.cc:301] APPLY: starting
0923 17:31:28.690641 (+ 104us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.690846 (+ 205us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.691030 (+ 184us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.691924 (+ 894us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.692161 (+ 237us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:28.692584 (+ 423us) write_op.cc:312] APPLY: finished
0923 17:31:28.692798 (+ 214us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.692956 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.693010 (+ 54us) write_op.cc:454] Released schema lock
0923 17:31:28.693044 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.693205 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.693228 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4869,"apply.run_cpu_time_us":2573,"apply.run_wall_time_us":2855,"num_ops":2,"prepare.queue_time_us":327972,"prepare.run_cpu_time_us":2736,"prepare.run_wall_time_us":21138,"replication_time_us":137597}]]}
I20260923 17:31:28.770080 17143 raft_consensus.cc:348] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.770778 17143 raft_consensus.cc:374] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.771072 17143 raft_consensus.cc:729] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.771659 17143 consensus_queue.cc:260] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.773514 17146 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:28.775727 17143 ts_tablet_manager.cc:1428] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260923 17:31:28.777881 17143 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.782907 17143 tablet_bootstrap.cc:654] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:28.787019 16972 consensus_peers.cc:487] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:28.697749 17144 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:28.801481 17144 ts_tablet_manager.cc:1397] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.192s user 0.004s sys 0.015s
I20260923 17:31:28.803906 17143 tablet_bootstrap.cc:492] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:28.804366 17143 ts_tablet_manager.cc:1397] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.003s
I20260923 17:31:28.803994 17144 raft_consensus.cc:348] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:28.804663 17144 raft_consensus.cc:374] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.804899 17144 raft_consensus.cc:729] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.805506 17144 consensus_queue.cc:260] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:28.705536 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 72010.3us
I20260923 17:31:28.807061 17144 ts_tablet_manager.cc:1428] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260923 17:31:28.807394 16678 leader_election.cc:302] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:28.807881 17144 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:28.812923 18092 raft_consensus.cc:2781] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:28.813295 18092 raft_consensus.cc:482] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:28.813608 18092 raft_consensus.cc:3037] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.817368 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.817947 17041 raft_consensus.cc:2454] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:28.818688 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.819176 17037 raft_consensus.cc:3037] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.821224 16792 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:28.824434 18092 raft_consensus.cc:504] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.826414 18092 leader_election.cc:288] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.830724 17037 raft_consensus.cc:2454] T 9f2e99ac8c0549b188cf0d872f33d1bf P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.834259 17144 tablet_bootstrap.cc:654] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.833374 17032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.837806 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "006895667f324d049debbebb132ed09b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.877213 17039 raft_consensus.cc:3037] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.838953 17034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.878518 17034 raft_consensus.cc:2454] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:28.879305 17032 raft_consensus.cc:3037] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.848975 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:28.881142 17041 raft_consensus.cc:3037] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.888170 17039 raft_consensus.cc:2454] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.790347 17146 tablet_bootstrap.cc:654] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:28.914467 17041 raft_consensus.cc:2454] T 107a073567314be996d6b8e3dbc47701 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:28.925201 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:28.933095 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 7c115e29673747f8b0712d441ac993fb
I20260923 17:31:28.934324 16387 catalog_manager.cc:6565] ReplaceTablet: tablet 7c115e29673747f8b0712d441ac993fb deleted and replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7
I20260923 17:31:28.934631 16387 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 229) took 3838 ms. Trace:
I20260923 17:31:28.934276 17143 raft_consensus.cc:348] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.935076 17143 raft_consensus.cc:374] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:28.935304 17143 raft_consensus.cc:729] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:28.937126 17143 consensus_queue.cc:260] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.939072 17143 ts_tablet_manager.cc:1428] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.134s user 0.004s sys 0.001s
I20260923 17:31:28.941220 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c115e29673747f8b0712d441ac993fb with delete_type TABLET_DATA_DELETED (replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:28.941291 16677 leader_election.cc:341] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:28.943370 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c115e29673747f8b0712d441ac993fb with delete_type TABLET_DATA_DELETED (replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:28.943411 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c115e29673747f8b0712d441ac993fb with delete_type TABLET_DATA_DELETED (replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:28.945156 17859 raft_consensus.cc:482] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.945756 16384 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 230) took 3653 ms. Trace:
I20260923 17:31:28.945608 17859 raft_consensus.cc:504] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.948678 17859 leader_election.cc:288] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.951308 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:28.934917 16387 rpcz_store.cc:276] 0923 17:31:25.095897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.085318 (+2989421us) service_pool.cc:227] Handling call
0923 17:31:28.934585 (+849267us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.567154 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.567590 (+ 436us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.567615 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:28.567631 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:28.567970 (+ 339us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.568006 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.568035 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.568277 (+ 242us) tablet.cc:699] Row locks acquired
0923 17:31:28.568290 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:28.568415 (+ 125us) write_op.cc:270] Start()
0923 17:31:28.568505 (+ 90us) write_op.cc:276] Timestamp: P: 1790184688568355 usec, L: 0
0923 17:31:28.568524 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.684571 (+116047us) log.cc:842] Serialized 631 byte log entry
0923 17:31:28.929303 (+244732us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.930855 (+ 1552us) write_op.cc:301] APPLY: starting
0923 17:31:28.930912 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.931017 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.931121 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.931557 (+ 436us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.932020 (+ 463us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:28.932060 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:28.932284 (+ 224us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.932453 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.932514 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:28.932542 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.932700 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.932726 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1391,"apply.run_cpu_time_us":2064,"apply.run_wall_time_us":2064,"num_ops":2,"prepare.queue_time_us":345353,"prepare.run_cpu_time_us":2585,"prepare.run_wall_time_us":126478,"replication_time_us":360690}]]}
I20260923 17:31:28.955909 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_044d6aa80d5d44339d4928ad987fd8c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:28.957479 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_044d6aa80d5d44339d4928ad987fd8c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:28.957477 17921 consensus_queue.cc:1035] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:28.946074 16384 rpcz_store.cc:276] 0923 17:31:25.292287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.539031 (+3246744us) service_pool.cc:227] Handling call
0923 17:31:28.545260 (+ 6229us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:28.548008 (+ 2748us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:28.945171 (+397163us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:28.945452 (+ 281us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:28.945696 (+ 244us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.715327 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.717017 (+ 1690us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.717044 (+ 27us) write_op.cc:435] Acquired schema lock
0923 17:31:28.717060 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:28.717375 (+ 315us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.717418 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.717441 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.717656 (+ 215us) tablet.cc:699] Row locks acquired
0923 17:31:28.717671 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:28.717792 (+ 121us) write_op.cc:270] Start()
0923 17:31:28.717890 (+ 98us) write_op.cc:276] Timestamp: P: 1790184688717722 usec, L: 0
0923 17:31:28.717910 (+ 20us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.718434 (+ 524us) log.cc:842] Serialized 433 byte log entry
0923 17:31:28.929668 (+211234us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.933028 (+ 3360us) write_op.cc:301] APPLY: starting
0923 17:31:28.933075 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.933165 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.933240 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.933702 (+ 462us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.933980 (+ 278us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:28.934021 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:28.934169 (+ 148us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.934423 (+ 254us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.934544 (+ 121us) write_op.cc:454] Released schema lock
0923 17:31:28.934603 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.934939 (+ 336us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.934988 (+ 49us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3254,"apply.run_cpu_time_us":2174,"apply.run_wall_time_us":2173,"num_ops":2,"prepare.queue_time_us":165904,"prepare.run_cpu_time_us":2178,"prepare.run_wall_time_us":14622,"replication_time_us":211693,"thread_start_us":9063,"threads_started":1}]]}
I20260923 17:31:28.962888 16384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_af6ea13bdd26434f9222c9255b68ee1e" } new_table_name: "default.Table_60e50e047c1244eab5a6b0552438404f" modify_external_catalogs: true
I20260923 17:31:28.964391 17859 raft_consensus.cc:482] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.965049 17859 raft_consensus.cc:504] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.968569 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c929b8beaaf4c7aa668852b63af2204" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.968801 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:28.970151 16827 leader_election.cc:302] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:28.971289 18037 raft_consensus.cc:2726] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:28.972868 17026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5b5504f7174bf483e9ddc79161079f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:28.973941 17026 raft_consensus.cc:2379] T 0e5b5504f7174bf483e9ddc79161079f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:28.976555 16827 leader_election.cc:302] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:28.977061 18012 raft_consensus.cc:482] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:28.977401 17143 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:28.977447 18012 raft_consensus.cc:504] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:28.980960 18012 leader_election.cc:288] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:28.982079 18037 raft_consensus.cc:3037] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:28.948532 18006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:28.995704 18006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012948ec54c1441381b43989690264ce, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.002449 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:29.009238 16677 leader_election.cc:341] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.005084 18006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:29.011184 18115 raft_consensus.cc:482] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.011653 17859 leader_election.cc:288] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.012003 18115 raft_consensus.cc:504] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.017180 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:29.018383 18115 leader_election.cc:288] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.018864 17026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be2b507dfd54f4eb821ea2911b4508f" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.021124 16827 leader_election.cc:302] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:29.021883 18115 raft_consensus.cc:2726] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.029079 17032 raft_consensus.cc:2454] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:29.038549 18012 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.039104 18012 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.042183 18012 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.043098 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:29.038924 18116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:29.046823 18024 raft_consensus.cc:2229] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:29.047416 18024 raft_consensus.cc:2258] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:29.047847 18116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:29.049494 18024 ts_tablet_manager.cc:1905] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:29.056602 16677 leader_election.cc:341] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.072674 18024 ts_tablet_manager.cc:1918] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:29.075976 16678 leader_election.cc:302] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.076189 16387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 231) took 3732 ms. Trace:
I20260923 17:31:29.076707 17921 raft_consensus.cc:2781] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:29.077046 17921 raft_consensus.cc:482] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:29.077351 17921 raft_consensus.cc:3037] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:29.083452 17921 raft_consensus.cc:504] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.085372 17921 leader_election.cc:288] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.092320 18105 tablet.cc:1722] T a6894a4ee5fe4884b5162d8055e766fe P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260923 17:31:29.076738 16387 rpcz_store.cc:276] 0923 17:31:25.343312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.955475 (+3612163us) service_pool.cc:227] Handling call
0923 17:31:28.957955 (+ 2480us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:28.959473 (+ 1518us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:29.074467 (+114994us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:29.075649 (+ 1182us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:29.076081 (+ 432us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.046918 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.047850 (+ 932us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.047897 (+ 47us) write_op.cc:435] Acquired schema lock
0923 17:31:29.047930 (+ 33us) tablet.cc:660] Decoding operations
0923 17:31:29.048512 (+ 582us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.048556 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.048585 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:29.048807 (+ 222us) tablet.cc:699] Row locks acquired
0923 17:31:29.048823 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:29.048948 (+ 125us) write_op.cc:270] Start()
0923 17:31:29.049038 (+ 90us) write_op.cc:276] Timestamp: P: 1790184689048885 usec, L: 0
0923 17:31:29.049059 (+ 21us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.049627 (+ 568us) log.cc:842] Serialized 433 byte log entry
0923 17:31:29.058024 (+ 8397us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.058316 (+ 292us) write_op.cc:301] APPLY: starting
0923 17:31:29.058368 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.058465 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.058562 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.058973 (+ 411us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.059103 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:29.059140 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:29.059309 (+ 169us) log.cc:842] Serialized 29 byte log entry
0923 17:31:29.059452 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.059511 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:29.059539 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:29.059693 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.059715 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":163,"apply.run_cpu_time_us":1538,"apply.run_wall_time_us":1537,"num_ops":2,"prepare.queue_time_us":86331,"prepare.run_cpu_time_us":3026,"prepare.run_wall_time_us":3026,"raft.queue_time_us":102,"raft.run_cpu_time_us":621,"raft.run_wall_time_us":621,"replication_time_us":8888}]]}
I20260923 17:31:29.101004 18006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:31:29.101773 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:29.102572 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:29.103150 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]). Expected version 1 got 0
I20260923 17:31:29.105181 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:29.122995 17146 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:29.123488 17146 ts_tablet_manager.cc:1397] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.350s user 0.008s sys 0.019s
I20260923 17:31:29.125653 17146 raft_consensus.cc:348] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.126441 17146 raft_consensus.cc:374] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.126720 17146 raft_consensus.cc:729] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.127547 17146 consensus_queue.cc:260] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.129839 17146 ts_tablet_manager.cc:1428] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
W20260923 17:31:29.130649 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 54764.2us
I20260923 17:31:29.113888 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:29.134122 18024 log.cc:1196] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/19ebfb3cfd3d49b39c7cf453199ac8bd
W20260923 17:31:29.135186 16677 leader_election.cc:341] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.121213 17143 tablet_bootstrap.cc:654] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:29.137205 18024 ts_tablet_manager.cc:1939] T 19ebfb3cfd3d49b39c7cf453199ac8bd P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:29.156936 18037 raft_consensus.cc:2726] T 0e5b5504f7174bf483e9ddc79161079f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.167119 17143 tablet_bootstrap.cc:492] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:29.167665 17143 ts_tablet_manager.cc:1397] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.194s user 0.009s sys 0.005s
I20260923 17:31:29.170433 17144 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:29.170058 17143 raft_consensus.cc:348] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.170713 17143 raft_consensus.cc:374] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.171000 17144 ts_tablet_manager.cc:1397] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.363s user 0.094s sys 0.105s
I20260923 17:31:29.170956 17143 raft_consensus.cc:729] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.173322 17144 raft_consensus.cc:348] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.174090 17144 raft_consensus.cc:374] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.174379 17144 raft_consensus.cc:729] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.175048 17144 consensus_queue.cc:260] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.176312 17143 consensus_queue.cc:260] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:29.196326 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 63875.3us
I20260923 17:31:29.203644 16678 leader_election.cc:302] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.204634 17143 ts_tablet_manager.cc:1428] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.037s user 0.003s sys 0.004s
I20260923 17:31:29.213568 17143 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:29.105249 16387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_67ae78aa612d4ebcbaf1dcb60c98be71" } modify_external_catalogs: true
I20260923 17:31:29.218331 16387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_67ae78aa612d4ebcbaf1dcb60c98be71" } modify_external_catalogs: true
I20260923 17:31:29.140494 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (table default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]) successfully deleted
W20260923 17:31:29.220428 16372 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 79972.5us
W20260923 17:31:29.221163 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 7c115e29673747f8b0712d441ac993fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c115e29673747f8b0712d441ac993fb already in progress: creating tablet
I20260923 17:31:29.142691 18068 raft_consensus.cc:482] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.227329 18068 raft_consensus.cc:504] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.229162 18068 leader_election.cc:288] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:29.180461 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.202898 18069 raft_consensus.cc:482] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.229696 18069 raft_consensus.cc:504] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.231334 18069 leader_election.cc:288] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:29.233505 17037 raft_consensus.cc:2454] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:29.235174 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:29.235483 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:29.239856 16972 leader_election.cc:341] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:29.241148 16972 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.244912 17146 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:29.248409 18020 ts_tablet_manager.cc:1918] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:29.249101 18020 log.cc:1196] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/261d1d03290546969a74ca60beb890eb
I20260923 17:31:29.249876 18020 ts_tablet_manager.cc:1939] T 261d1d03290546969a74ca60beb890eb P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:29.250895 18020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 215) took 1518 ms. Trace:
I20260923 17:31:29.223069 17144 ts_tablet_manager.cc:1428] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260923 17:31:29.230265 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:29.219944 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:29.232939 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:29.260241 18012 raft_consensus.cc:2781] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:29.260614 18012 raft_consensus.cc:482] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:29.260905 18012 raft_consensus.cc:3037] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:29.260882 16972 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.261332 16972 leader_election.cc:302] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:29.262399 16972 leader_election.cc:341] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.262734 16972 leader_election.cc:302] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.263375 18068 raft_consensus.cc:2726] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.263914 18068 raft_consensus.cc:2726] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.264106 16382 catalog_manager.cc:5495] T 3208c90d96f349988acb6705f2f438d5 P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.265774 16382 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:29.269963 18012 raft_consensus.cc:504] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.274108 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:29.275522 16677 leader_election.cc:341] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.275651 17144 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:29.277158 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 73591.3us
I20260923 17:31:29.277814 16678 leader_election.cc:302] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.278487 17884 raft_consensus.cc:2781] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:29.278754 16678 leader_election.cc:302] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.279021 17884 raft_consensus.cc:686] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:29.279667 16678 leader_election.cc:302] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.280144 17884 consensus_queue.cc:237] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.282444 17921 raft_consensus.cc:2781] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:29.283159 17921 raft_consensus.cc:686] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:29.283329 17753 raft_consensus.cc:2781] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:29.284075 17753 raft_consensus.cc:686] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:29.284376 17921 consensus_queue.cc:237] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.285100 17753 consensus_queue.cc:237] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.293306 16892 raft_consensus.cc:3037] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:29.296733 16972 consensus_peers.cc:487] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19ebfb3cfd3d49b39c7cf453199ac8bd. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:29.297554 18012 leader_election.cc:288] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.305809 17143 tablet_bootstrap.cc:654] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:29.336606 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b77ef164608f4ced8b1c2ecab6f88c1a (table default.Table_b0ce21ff6a6e46c8a4a37cdc3ab4f877 [id=a67f05c6ae754dcba81905a4dd293a0f], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet b77ef164608f4ced8b1c2ecab6f88c1a consensus state has no leader
I20260923 17:31:29.251251 18020 rpcz_store.cc:276] 0923 17:31:27.732416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.732565 (+ 149us) service_pool.cc:227] Handling call
0923 17:31:28.022724 (+290159us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:28.022759 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:28.616329 (+593570us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:29.248137 (+631808us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:29.250834 (+ 2697us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":284542,"tablet-delete.queue_time_us":289561}
I20260923 17:31:29.338384 17143 tablet_bootstrap.cc:492] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:29.338831 17143 ts_tablet_manager.cc:1397] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.125s user 0.009s sys 0.003s
I20260923 17:31:29.339447 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 261d1d03290546969a74ca60beb890eb (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
I20260923 17:31:29.340921 16886 raft_consensus.cc:3037] T 1be2b507dfd54f4eb821ea2911b4508f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:29.352454 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 72831.9us
I20260923 17:31:29.370751 18012 raft_consensus.cc:482] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.371311 18012 raft_consensus.cc:504] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.373386 17146 tablet_bootstrap.cc:654] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:29.373402 18012 leader_election.cc:288] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.375523 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:29.377774 16677 leader_election.cc:341] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.379032 17143 raft_consensus.cc:348] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.379711 17143 raft_consensus.cc:374] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.379940 17143 raft_consensus.cc:729] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.388504 18020 tablet_replica.cc:330] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:29.392010 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 4cd483bcad624c9a86503bdf15b9d6f8
I20260923 17:31:29.393890 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:29 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:29.395833 16387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 233) took 1893 ms. Trace:
I20260923 17:31:29.395916 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:29 UTC) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:29.397528 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 4cd483bcad624c9a86503bdf15b9d6f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cd483bcad624c9a86503bdf15b9d6f8 already in progress: creating tablet
I20260923 17:31:29.397796 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:29 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:29.396270 16387 rpcz_store.cc:276] 0923 17:31:27.502520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:29.104952 (+1602432us) service_pool.cc:227] Handling call
0923 17:31:29.218772 (+113820us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:29.221791 (+ 3019us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:29.221947 (+ 156us) catalog_manager.cc:2761] Locking tablets
0923 17:31:29.222582 (+ 635us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:29.245686 (+ 23104us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:29.391496 (+145810us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:29.391926 (+ 430us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:29.395761 (+ 3835us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.224501 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.224911 (+ 410us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.224937 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:29.224955 (+ 18us) tablet.cc:660] Decoding operations
0923 17:31:29.225273 (+ 318us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.225319 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.225343 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:29.225624 (+ 281us) tablet.cc:699] Row locks acquired
0923 17:31:29.225640 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:29.225750 (+ 110us) write_op.cc:270] Start()
0923 17:31:29.225833 (+ 83us) write_op.cc:276] Timestamp: P: 1790184689225685 usec, L: 0
0923 17:31:29.225864 (+ 31us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.226339 (+ 475us) log.cc:842] Serialized 734 byte log entry
0923 17:31:29.237814 (+ 11475us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.238271 (+ 457us) write_op.cc:301] APPLY: starting
0923 17:31:29.238320 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.238428 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.238515 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.238740 (+ 225us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.238902 (+ 162us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:29.238940 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:29.239114 (+ 174us) log.cc:842] Serialized 29 byte log entry
0923 17:31:29.239252 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.239318 (+ 66us) write_op.cc:454] Released schema lock
0923 17:31:29.239347 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:29.239522 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.239544 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":304,"apply.run_cpu_time_us":1438,"apply.run_wall_time_us":1439,"num_ops":2,"prepare.queue_time_us":346,"prepare.run_cpu_time_us":2167,"prepare.run_wall_time_us":20236,"raft.queue_time_us":82,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":585,"replication_time_us":11880}]]}
I20260923 17:31:29.400591 17143 consensus_queue.cc:260] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.416862 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_c0f60278d3e544bcbb2362a9aed27cd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:29.417505 18020 raft_consensus.cc:2229] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:29.419008 18020 raft_consensus.cc:2258] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:29.355594 16678 leader_election.cc:302] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.423436 17921 raft_consensus.cc:2781] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:29.424086 17921 raft_consensus.cc:686] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:29.425297 17921 consensus_queue.cc:237] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.428934 16892 raft_consensus.cc:3037] T e09c716342a6445a8edab130fd78a8e3 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:29.435124 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0f60278d3e544bcbb2362a9aed27cd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:29.440176 18115 raft_consensus.cc:482] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.440971 18115 raft_consensus.cc:504] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.445842 17143 ts_tablet_manager.cc:1428] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.107s user 0.006s sys 0.003s
I20260923 17:31:29.451298 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.453158 17143 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:29.456539 18115 leader_election.cc:288] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.487493 16387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3641832c7727419ba0ccb14409b39259"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:29.440145 17144 tablet_bootstrap.cc:654] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:29.499862 16387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3641832c7727419ba0ccb14409b39259 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:29.423493 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 67996.9us
I20260923 17:31:29.445706 16745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2e5bc4ae186404c896814b95d6cc7ef" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:29.504213 18020 ts_tablet_manager.cc:1905] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:29.508817 16826 leader_election.cc:302] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:29.510022 18115 raft_consensus.cc:2726] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.479118 17146 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:29.514408 17146 ts_tablet_manager.cc:1397] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.269s user 0.005s sys 0.007s
I20260923 17:31:29.455714 18021 ts_tablet_manager.cc:1918] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:29.515451 18021 log.cc:1196] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:29.516502 18021 ts_tablet_manager.cc:1939] T 19ebfb3cfd3d49b39c7cf453199ac8bd P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:29.517349 18021 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 259) took 1063 ms. Trace:
I20260923 17:31:29.534370 16387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_96a86405cb7a4a9493b667daf383bf1c" } new_table_name: "default.Table_18752eef74c04de2852506a9255132a7" modify_external_catalogs: true
I20260923 17:31:29.494123 16379 catalog_manager.cc:5495] T e4a2a0d1f4344ad69eb2c0a4e208839a P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.543155 17143 tablet_bootstrap.cc:654] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:29.545146 16379 catalog_manager.cc:5495] T 7efa62885c564a9bb1ca1b9beacf6e73 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.553098 18021 rpcz_store.cc:276] 0923 17:31:28.454245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.454402 (+ 157us) service_pool.cc:227] Handling call
0923 17:31:28.556212 (+101810us) ts_tablet_manager.cc:1145] Deleting tablet 19ebfb3cfd3d49b39c7cf453199ac8bd
0923 17:31:28.564884 (+ 8672us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:29.207766 (+642882us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:29.455323 (+247557us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:29.517295 (+ 61972us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":70406}
I20260923 17:31:29.554653 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (table default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]) successfully deleted
I20260923 17:31:29.557170 16379 catalog_manager.cc:5495] T 1822edfa26b045d7a15074a4bd5cbf5e P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.557909 17144 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:29.558306 17144 ts_tablet_manager.cc:1397] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.283s user 0.011s sys 0.001s
I20260923 17:31:29.560616 17144 raft_consensus.cc:348] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.561223 17144 raft_consensus.cc:374] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.561442 17144 raft_consensus.cc:729] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.538700 16678 leader_election.cc:302] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:29.562426 17884 raft_consensus.cc:2781] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:29.562719 17884 raft_consensus.cc:482] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:29.563001 17884 raft_consensus.cc:3037] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:29.569433 17884 raft_consensus.cc:504] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.571113 17884 leader_election.cc:288] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.555961 17146 raft_consensus.cc:348] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.571872 17146 raft_consensus.cc:374] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.573527 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.574115 17037 raft_consensus.cc:2454] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:29.574908 17859 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.575008 17143 tablet_bootstrap.cc:492] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:29.575666 17143 ts_tablet_manager.cc:1397] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.129s user 0.001s sys 0.012s
I20260923 17:31:29.575352 17859 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.577224 17859 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:29.578944 17143 raft_consensus.cc:348] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.579576 17143 raft_consensus.cc:374] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:29.579802 17143 raft_consensus.cc:729] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:29.580173 16827 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:29.579934 16379 catalog_manager.cc:5495] T e329dd46a3a245b6826b01ac8de56621 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.580966 16379 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:29.581784 16379 catalog_manager.cc:5495] T 47a0f16066164132a26927f5a830e677 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.583479 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:29.584424 17037 raft_consensus.cc:3037] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:29.587833 17143 consensus_queue.cc:260] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.589478 17143 ts_tablet_manager.cc:1428] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20260923 17:31:29.591451 17143 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:29.595817 16379 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
W20260923 17:31:29.585995 16387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e6d9fad7151a4551ba8f2b0e25525cc7 has no consensus state
I20260923 17:31:29.599016 17143 tablet_bootstrap.cc:654] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:29.602190 16379 catalog_manager.cc:5495] T 6eb728f7f83e4975823e338a4568f796 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.615231 16379 catalog_manager.cc:5495] T 6a90f7348a964a7f871e93dde7ab82f2 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:29.622177 16379 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:29.650434 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:29.572129 17146 raft_consensus.cc:729] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:29.664525 17146 consensus_queue.cc:260] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:29.708688 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 7c115e29673747f8b0712d441ac993fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c115e29673747f8b0712d441ac993fb already in progress: creating tablet
W20260923 17:31:29.713760 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 4cd483bcad624c9a86503bdf15b9d6f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cd483bcad624c9a86503bdf15b9d6f8 already in progress: creating tablet
I20260923 17:31:29.718442 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:29.719046 17036 raft_consensus.cc:3037] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:29.725500 17036 raft_consensus.cc:2454] T 254793351e5245af9a4fa1642cad5e82 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:29.731088 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.735500 17036 raft_consensus.cc:2454] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:29.735347 16381 catalog_manager.cc:5495] T 897f08f065fb4bc4bfa03853e4f696bc P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:29.749358 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:29.755620 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:29.756440 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:29.758230 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]). Expected version 1 got 0
I20260923 17:31:29.760684 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:29.767575 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
W20260923 17:31:29.753253 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 95894.4us
I20260923 17:31:29.774832 17037 raft_consensus.cc:2454] T c1d71b22338241839b184e50d7fdd8f6 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:29.777273 16381 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
W20260923 17:31:29.783422 16677 leader_election.cc:341] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:29.777271 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 85423.9us
I20260923 17:31:29.811064 17146 ts_tablet_manager.cc:1428] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.296s user 0.006s sys 0.000s
I20260923 17:31:29.812084 17146 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:29.820703 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:29.821570 17037 raft_consensus.cc:3037] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:29.561928 17144 consensus_queue.cc:260] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:29.834681 17144 ts_tablet_manager.cc:1428] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.276s user 0.006s sys 0.000s
I20260923 17:31:29.835697 17144 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:29.840368 18152 raft_consensus.cc:482] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.840895 18152 raft_consensus.cc:504] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.843554 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:29.844772 17859 raft_consensus.cc:482] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.845180 17859 raft_consensus.cc:504] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:29.846915 17859 leader_election.cc:288] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:29.848822 16972 leader_election.cc:341] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:29.837069 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:29.853344 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]). Expected version 1 got 0
I20260923 17:31:29.861428 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:29.862286 17036 raft_consensus.cc:2379] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:29.865213 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:29.867707 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:29.868443 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
W20260923 17:31:29.881067 16826 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:29.881456 16826 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:29.882288 17859 raft_consensus.cc:2726] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.928637 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]). Expected version 1 got 0
I20260923 17:31:29.929746 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:29.937134 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:29.938266 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:29.939011 16386 catalog_manager.cc:5495] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to e5c071cdf65042a59cad226a15a5532c (127.15.246.131). New cstate: current_term: 1 leader_uuid: "e5c071cdf65042a59cad226a15a5532c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:29.939750 18152 leader_election.cc:288] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:29.948289 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:29.957160 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:29.965068 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:29.970903 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e92cfe501505458ea46da095a8c0d30a (table default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4]). Expected version 2 got 0
I20260923 17:31:29.971755 16386 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:29.998003 18092 raft_consensus.cc:482] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:29.998435 18092 raft_consensus.cc:504] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.001828 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7c8e360d954ce78913e78c8662c341" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:30.010413 18092 leader_election.cc:288] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.033299 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:30.038712 16886 raft_consensus.cc:3037] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.040668 17143 tablet_bootstrap.cc:492] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:30.041081 17143 ts_tablet_manager.cc:1397] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.450s user 0.101s sys 0.168s
I20260923 17:31:30.043649 17143 raft_consensus.cc:348] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.045557 16884 raft_consensus.cc:3037] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.048569 17143 raft_consensus.cc:374] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.048890 17143 raft_consensus.cc:729] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.049517 17143 consensus_queue.cc:260] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.051498 17143 ts_tablet_manager.cc:1428] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260923 17:31:30.052394 17143 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:30.056859 16678 leader_election.cc:302] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.058456 17753 raft_consensus.cc:2781] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:30.058990 17753 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:30.059531 17753 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.062636 17143 tablet_bootstrap.cc:654] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:30.064416 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 183381us
I20260923 17:31:30.067063 17884 consensus_queue.cc:1035] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.071650 16678 leader_election.cc:302] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.081619 18012 raft_consensus.cc:2781] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.082556 18012 raft_consensus.cc:686] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.084204 18012 consensus_queue.cc:237] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.094542 17144 tablet_bootstrap.cc:654] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.087708 17753 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.100450 16678 leader_election.cc:302] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.102768 18092 raft_consensus.cc:2781] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:30.103473 18092 raft_consensus.cc:482] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:30.104068 18092 raft_consensus.cc:3037] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.108125 18158 consensus_queue.cc:1035] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.104007 17753 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:30.121436 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 56072.1us
W20260923 17:31:30.122278 16677 consensus_peers.cc:487] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:30.122366 16678 leader_election.cc:302] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.127633 17884 raft_consensus.cc:2781] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.130913 18158 consensus_queue.cc:1035] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.128101 17884 raft_consensus.cc:686] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.133145 17884 consensus_queue.cc:237] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:30.039456 16972 leader_election.cc:341] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.145236 16972 leader_election.cc:302] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:30.146113 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 106703us
I20260923 17:31:30.147742 18068 raft_consensus.cc:2726] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:29.983268 17037 raft_consensus.cc:2454] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.162911 18012 consensus_queue.cc:1035] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.091675 17146 tablet_bootstrap.cc:654] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.168110 17143 tablet_bootstrap.cc:492] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:30.168651 17143 ts_tablet_manager.cc:1397] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.116s user 0.027s sys 0.059s
I20260923 17:31:30.169595 18092 raft_consensus.cc:504] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.118546 18020 ts_tablet_manager.cc:1918] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:30.172694 18020 log.cc:1196] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:30.173914 18020 ts_tablet_manager.cc:1939] T 19ebfb3cfd3d49b39c7cf453199ac8bd P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:30.176836 17143 raft_consensus.cc:348] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.177496 17143 raft_consensus.cc:374] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.177742 17143 raft_consensus.cc:729] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.178313 17143 consensus_queue.cc:260] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.182664 17143 ts_tablet_manager.cc:1428] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260923 17:31:30.183570 17143 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:30.188979 17143 tablet_bootstrap.cc:654] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.192409 18092 leader_election.cc:288] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.200328 18020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 216) took 1723 ms. Trace:
I20260923 17:31:30.201051 18020 rpcz_store.cc:276] 0923 17:31:28.459520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.459667 (+ 147us) service_pool.cc:227] Handling call
0923 17:31:29.388148 (+928481us) ts_tablet_manager.cc:1145] Deleting tablet 19ebfb3cfd3d49b39c7cf453199ac8bd
0923 17:31:29.388368 (+ 220us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:29.756611 (+368243us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:30.118286 (+361675us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:30.182827 (+ 64541us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26562,"tablet-delete.queue_time_us":815373}
I20260923 17:31:30.203239 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 19ebfb3cfd3d49b39c7cf453199ac8bd (table default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]) successfully deleted
I20260923 17:31:30.204250 18020 tablet_replica.cc:330] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:30.206063 18020 raft_consensus.cc:2229] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:30.210783 18020 raft_consensus.cc:2258] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260923 17:31:30.212155 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 51029.3us
W20260923 17:31:30.214238 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 83675us
I20260923 17:31:30.218053 18020 ts_tablet_manager.cc:1905] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:30.231297 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:30.238389 17144 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:30.238919 17144 ts_tablet_manager.cc:1397] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.403s user 0.005s sys 0.005s
I20260923 17:31:30.241596 17144 raft_consensus.cc:348] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.242313 17144 raft_consensus.cc:374] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.242554 17144 raft_consensus.cc:729] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.243127 17144 consensus_queue.cc:260] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.246888 17146 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:30.248759 16826 leader_election.cc:302] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:30.249615 16714 tablet_service.cc:1467] Processing CreateTablet for tablet d15e626ea47f435b8cc75d91d3d04900 (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.249948 17859 raft_consensus.cc:2726] T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:30.250984 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15e626ea47f435b8cc75d91d3d04900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.252143 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:30.261557 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:30.264534 17146 ts_tablet_manager.cc:1397] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.453s user 0.012s sys 0.000s
I20260923 17:31:30.267176 17146 raft_consensus.cc:348] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:30.267395 18132 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 2212082103761160720) tablet 00000000000000000000000000000000 cur holder 0x7b5800031800
I20260923 17:31:30.267841 17146 raft_consensus.cc:374] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.268090 17146 raft_consensus.cc:729] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
W20260923 17:31:30.268034 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 51375.7us
I20260923 17:31:30.272526 16865 tablet_service.cc:1467] Processing CreateTablet for tablet d15e626ea47f435b8cc75d91d3d04900 (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.273638 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15e626ea47f435b8cc75d91d3d04900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.282941 18020 ts_tablet_manager.cc:1918] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:30.285049 17146 consensus_queue.cc:260] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.292161 17018 tablet_service.cc:1467] Processing CreateTablet for tablet d15e626ea47f435b8cc75d91d3d04900 (DEFAULT_TABLE table=default.Table_828238fcfb3446679de0ee7995c81b99 [id=375d499e0453467b80889f37b6101554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.294250 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15e626ea47f435b8cc75d91d3d04900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.297968 16383 catalog_manager.cc:5495] T e09c716342a6445a8edab130fd78a8e3 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:30.299470 17146 ts_tablet_manager.cc:1428] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260923 17:31:30.299875 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:30.300927 17146 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:30.311323 18020 log.cc:1196] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/7c115e29673747f8b0712d441ac993fb
I20260923 17:31:30.312162 18020 ts_tablet_manager.cc:1939] T 7c115e29673747f8b0712d441ac993fb P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:30.313315 17143 tablet_bootstrap.cc:492] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:30.313912 17143 ts_tablet_manager.cc:1397] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.130s user 0.013s sys 0.005s
I20260923 17:31:30.313942 18020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 217) took 1373 ms. Trace:
I20260923 17:31:30.317559 17143 raft_consensus.cc:348] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.318204 17143 raft_consensus.cc:374] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.318439 17143 raft_consensus.cc:729] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.319180 18092 raft_consensus.cc:482] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:30.319660 18092 raft_consensus.cc:504] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:30.321354 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 52206.4us
I20260923 17:31:30.324602 18161 consensus_queue.cc:1035] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:30.326367 18092 leader_election.cc:288] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.327562 16678 leader_election.cc:302] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.327764 17143 consensus_queue.cc:260] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.329408 17143 ts_tablet_manager.cc:1428] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260923 17:31:30.330121 17753 raft_consensus.cc:2781] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.330261 17143 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:30.330536 17753 raft_consensus.cc:686] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.331220 17753 consensus_queue.cc:237] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.341070 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:30.315788 18020 rpcz_store.cc:276] 0923 17:31:28.940939 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.941069 (+ 130us) service_pool.cc:227] Handling call
0923 17:31:30.204071 (+1263002us) ts_tablet_manager.cc:1145] Deleting tablet 7c115e29673747f8b0712d441ac993fb
0923 17:31:30.204116 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:30.274083 (+ 69967us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:30.282681 (+ 8598us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:30.313891 (+ 31210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":611584,"tablet-delete.queue_time_us":1262471}
W20260923 17:31:30.345956 16677 leader_election.cc:341] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.368430 16383 catalog_manager.cc:5495] T c7e6178ab07245e3a687d53eb2ff03ab P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:30.371697 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
W20260923 17:31:30.372296 16677 leader_election.cc:341] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:30.373118 16677 consensus_peers.cc:487] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:30.373639 16677 leader_election.cc:341] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.379796 16383 catalog_manager.cc:5495] T 1be2b507dfd54f4eb821ea2911b4508f P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:30.381794 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:30.355224 17144 ts_tablet_manager.cc:1428] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.116s user 0.005s sys 0.000s
I20260923 17:31:30.357545 16792 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:30.386194 17144 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:30.394495 17032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:30.395213 17032 raft_consensus.cc:3037] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.399775 17034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.400440 17034 raft_consensus.cc:2454] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:30.403059 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.403671 17037 raft_consensus.cc:2454] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:30.404263 17033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:30.404776 17033 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:30.405210 16677 leader_election.cc:341] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.405308 18158 consensus_queue.cc:1035] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.406656 17753 consensus_queue.cc:1035] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:30.412783 16678 leader_election.cc:302] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.414671 17753 raft_consensus.cc:2781] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:30.415096 17753 raft_consensus.cc:482] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:30.415556 17753 raft_consensus.cc:3037] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.417872 16678 leader_election.cc:302] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.418642 18158 raft_consensus.cc:2781] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:30.418988 18158 raft_consensus.cc:482] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:30.419267 18158 raft_consensus.cc:3037] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.427570 17033 raft_consensus.cc:2454] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.428757 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 7c115e29673747f8b0712d441ac993fb (table default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]) successfully deleted
I20260923 17:31:30.433521 18012 consensus_queue.cc:1035] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.437749 17144 tablet_bootstrap.cc:654] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.443077 17753 raft_consensus.cc:504] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.448033 17753 leader_election.cc:288] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.449875 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:30.451658 16383 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:30.451943 18158 raft_consensus.cc:504] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.454541 17144 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:30.454998 17144 ts_tablet_manager.cc:1397] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.007s
I20260923 17:31:30.460929 17144 raft_consensus.cc:348] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.461675 17144 raft_consensus.cc:374] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.461946 17144 raft_consensus.cc:729] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.462539 17144 consensus_queue.cc:260] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.464743 17144 ts_tablet_manager.cc:1428] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.009s user 0.001s sys 0.005s
I20260923 17:31:30.465562 17144 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:30.470013 18166 log.cc:924] Time spent T 8e7c8e360d954ce78913e78c8662c341 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260923 17:31:30.472340 18158 leader_election.cc:288] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.490823 17144 tablet_bootstrap.cc:654] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.491271 16892 raft_consensus.cc:3037] T 006895667f324d049debbebb132ed09b P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.491647 17143 tablet_bootstrap.cc:654] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.492863 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:30.500825 16884 raft_consensus.cc:3037] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.506327 16889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
W20260923 17:31:30.522778 18170 log.cc:924] Time spent T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217: Append to log took a long time: real 0.130s user 0.000s sys 0.000s
I20260923 17:31:30.431670 18020 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 218) took 1034 ms. Trace:
I20260923 17:31:30.523820 18020 rpcz_store.cc:276] 0923 17:31:29.397367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:29.397497 (+ 130us) service_pool.cc:227] Handling call
0923 17:31:30.428659 (+1031162us) ts_tablet_manager.cc:1145] Deleting tablet 4cd483bcad624c9a86503bdf15b9d6f8
0923 17:31:30.431265 (+ 2606us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:30.431609 (+ 344us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1030481}
I20260923 17:31:30.443707 17032 raft_consensus.cc:2454] T ef890c51e92244f4b97c490f24250679 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.449291 17031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
W20260923 17:31:30.528918 16373 catalog_manager.cc:4821] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 4cd483bcad624c9a86503bdf15b9d6f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cd483bcad624c9a86503bdf15b9d6f8 already in progress: creating tablet
I20260923 17:31:30.531252 17031 raft_consensus.cc:3037] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:30.530434 16677 leader_election.cc:341] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:30.535030 16677 leader_election.cc:341] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.480194 17023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:30.536543 17023 raft_consensus.cc:3037] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.530814 16678 leader_election.cc:302] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.539201 18161 raft_consensus.cc:2781] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.539698 18161 raft_consensus.cc:686] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.541049 18161 consensus_queue.cc:237] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.568013 17143 tablet_bootstrap.cc:492] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:30.568816 17143 ts_tablet_manager.cc:1397] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.239s user 0.004s sys 0.011s
I20260923 17:31:30.571133 16678 leader_election.cc:302] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.573105 17884 raft_consensus.cc:2781] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.573586 17884 raft_consensus.cc:686] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.574342 17884 consensus_queue.cc:237] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.582484 18187 raft_consensus.cc:482] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:30.583048 18187 raft_consensus.cc:504] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.586230 18187 leader_election.cc:288] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.589856 18182 raft_consensus.cc:482] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:30.590679 18182 raft_consensus.cc:504] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.593114 17144 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:30.593604 17144 ts_tablet_manager.cc:1397] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.128s user 0.011s sys 0.010s
W20260923 17:31:30.562021 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 220947us
I20260923 17:31:30.602020 18182 leader_election.cc:288] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.604074 17144 raft_consensus.cc:348] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.604826 17144 raft_consensus.cc:374] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.605077 17144 raft_consensus.cc:729] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.629912 17144 consensus_queue.cc:260] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.632901 17144 ts_tablet_manager.cc:1428] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.037s user 0.005s sys 0.002s
I20260923 17:31:30.633970 17144 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:30.635732 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:30.637914 18161 consensus_queue.cc:1035] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:30.638487 18187 consensus_queue.cc:1035] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:30.648066 18182 raft_consensus.cc:482] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:30.649004 18182 raft_consensus.cc:504] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.606434 17143 raft_consensus.cc:348] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.617008 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba0ed72a55d74ab5b8b22fba90848094" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:30.652292 17143 raft_consensus.cc:374] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.652611 17143 raft_consensus.cc:729] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.652321 18182 leader_election.cc:288] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.653285 17143 consensus_queue.cc:260] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.654876 17143 ts_tablet_manager.cc:1428] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.086s user 0.003s sys 0.002s
I20260923 17:31:30.655702 17143 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:30.677124 16677 leader_election.cc:341] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.681209 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107a073567314be996d6b8e3dbc47701" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.555847 17023 raft_consensus.cc:2454] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.689555 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.691741 16827 leader_election.cc:302] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:30.693904 17025 raft_consensus.cc:2379] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:30.696604 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba0ed72a55d74ab5b8b22fba90848094" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.697404 17036 raft_consensus.cc:2454] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:30.711887 16678 leader_election.cc:302] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.713127 16678 leader_election.cc:302] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.715519 18012 raft_consensus.cc:2781] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.715961 18012 raft_consensus.cc:686] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.716672 18167 raft_consensus.cc:2726] T 107a073567314be996d6b8e3dbc47701 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:30.716786 18012 consensus_queue.cc:237] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.719717 18161 consensus_queue.cc:1035] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:30.723641 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (table default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e6d9fad7151a4551ba8f2b0e25525cc7 has no consensus state
I20260923 17:31:30.726608 18158 raft_consensus.cc:2781] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:30.727280 18158 raft_consensus.cc:482] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:30.727901 18158 raft_consensus.cc:3037] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.741386 17143 tablet_bootstrap.cc:654] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.747066 17144 tablet_bootstrap.cc:654] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.765079 18158 raft_consensus.cc:504] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:30.703212 17031 raft_consensus.cc:2454] T 8a2f1143dcd64b2dbd25e17787096903 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.780632 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dbd8eb0ab9243bd81c3d6a893f18aea" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:30.783236 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba0ed72a55d74ab5b8b22fba90848094" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:30.788146 16884 raft_consensus.cc:3037] T 31a83c7168a64b64bd224ef5a4c6939e P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.805522 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba0ed72a55d74ab5b8b22fba90848094" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:30.808112 16886 raft_consensus.cc:3037] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.809839 16826 leader_election.cc:302] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:30.784108 17025 raft_consensus.cc:3037] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:30.829483 16678 leader_election.cc:302] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:30.830653 18187 raft_consensus.cc:2781] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:30.842116 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 20382820ba9d490cab3299a29c378d2f (DEFAULT_TABLE table=default.Table_3641832c7727419ba0ccb14409b39259 [id=6da60586785c487a9660176c10fe5439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.843263 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20382820ba9d490cab3299a29c378d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.846735 17143 tablet_bootstrap.cc:492] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:30.847169 17143 ts_tablet_manager.cc:1397] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.192s user 0.011s sys 0.001s
I20260923 17:31:30.851037 17143 raft_consensus.cc:348] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.851752 17143 raft_consensus.cc:374] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.851994 17143 raft_consensus.cc:729] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.853040 17143 consensus_queue.cc:260] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.855173 17143 ts_tablet_manager.cc:1428] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260923 17:31:30.856127 17143 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:30.861581 17146 tablet_bootstrap.cc:654] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:30.827310 18158 leader_election.cc:288] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:30.872126 18195 log.cc:924] Time spent T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260923 17:31:30.872068 18182 raft_consensus.cc:2726] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:30.882076 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 20382820ba9d490cab3299a29c378d2f (DEFAULT_TABLE table=default.Table_3641832c7727419ba0ccb14409b39259 [id=6da60586785c487a9660176c10fe5439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.883241 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20382820ba9d490cab3299a29c378d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.871783 17144 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:30.887878 17144 ts_tablet_manager.cc:1397] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.254s user 0.005s sys 0.008s
I20260923 17:31:30.886960 18187 raft_consensus.cc:686] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:30.886752 16714 tablet_service.cc:1467] Processing CreateTablet for tablet 20382820ba9d490cab3299a29c378d2f (DEFAULT_TABLE table=default.Table_3641832c7727419ba0ccb14409b39259 [id=6da60586785c487a9660176c10fe5439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.890226 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20382820ba9d490cab3299a29c378d2f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:30.894361 16677 leader_election.cc:341] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:30.896108 18187 consensus_queue.cc:237] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.898304 18158 consensus_queue.cc:1035] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:30.900058 17144 raft_consensus.cc:348] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.900836 17144 raft_consensus.cc:374] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:30.901450 17144 raft_consensus.cc:729] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:30.907235 17144 consensus_queue.cc:260] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.912320 16869 tablet_service.cc:1467] Processing CreateTablet for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (DEFAULT_TABLE table=default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.913424 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d9fad7151a4551ba8f2b0e25525cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.916492 17143 tablet_bootstrap.cc:654] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:30.926297 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 94836.5us
I20260923 17:31:30.929036 16726 tablet_service.cc:1467] Processing CreateTablet for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (DEFAULT_TABLE table=default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.930059 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d9fad7151a4551ba8f2b0e25525cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.944247 16865 tablet_service.cc:1467] Processing CreateTablet for tablet fdcca5a4d30c413084f58b669cfbdcd8 (DEFAULT_TABLE table=default.Table_6df3e9d4c07446dd9e203188dcb3329c [id=4bf0e008458441ba8e2725d1949351c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.945436 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcca5a4d30c413084f58b669cfbdcd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.955107 17012 tablet_service.cc:1467] Processing CreateTablet for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (DEFAULT_TABLE table=default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:30.956838 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d9fad7151a4551ba8f2b0e25525cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:30.959569 18167 raft_consensus.cc:482] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:30.960254 18167 raft_consensus.cc:504] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:30.966218 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:30.973423 18167 leader_election.cc:288] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:30.988137 17025 raft_consensus.cc:2454] T ba0ed72a55d74ab5b8b22fba90848094 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:30.994930 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:31.000576 17021 tablet_service.cc:1467] Processing CreateTablet for tablet fdcca5a4d30c413084f58b669cfbdcd8 (DEFAULT_TABLE table=default.Table_6df3e9d4c07446dd9e203188dcb3329c [id=4bf0e008458441ba8e2725d1949351c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.000871 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 760bad8ccecc47418ccbeb8960c018db (DEFAULT_TABLE table=default.Table_c0f60278d3e544bcbb2362a9aed27cd0 [id=7ed1fb5bfa8b4c43bcc312e3a8df7ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.002055 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcca5a4d30c413084f58b669cfbdcd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.016463 17146 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:31.017163 17146 ts_tablet_manager.cc:1397] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.716s user 0.008s sys 0.014s
W20260923 17:31:31.018321 17039 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.012s sys 0.016s
W20260923 17:31:31.032516 17039 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.516s user 0.013s sys 0.016s
I20260923 17:31:31.044692 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 760bad8ccecc47418ccbeb8960c018db (DEFAULT_TABLE table=default.Table_c0f60278d3e544bcbb2362a9aed27cd0 [id=7ed1fb5bfa8b4c43bcc312e3a8df7ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.042905 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 43bce0fe5fdf459daeeda19dca2665d2 (DEFAULT_TABLE table=default.Table_37d4870a361e4247a261b101b89d1ffb [id=b1f352b253e74ec9a5d1ffe40cd39ce0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:31.013274 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 87925.8us
I20260923 17:31:31.054073 17144 ts_tablet_manager.cc:1428] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.166s user 0.007s sys 0.000s
W20260923 17:31:31.056499 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 129455us
I20260923 17:31:31.056915 16716 tablet_service.cc:1467] Processing CreateTablet for tablet fdcca5a4d30c413084f58b669cfbdcd8 (DEFAULT_TABLE table=default.Table_6df3e9d4c07446dd9e203188dcb3329c [id=4bf0e008458441ba8e2725d1949351c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.057942 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcca5a4d30c413084f58b669cfbdcd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.057974 16678 leader_election.cc:302] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters: c36e79694e154f84a46980719217d7f2
I20260923 17:31:31.059634 16720 tablet_service.cc:1467] Processing CreateTablet for tablet 760bad8ccecc47418ccbeb8960c018db (DEFAULT_TABLE table=default.Table_c0f60278d3e544bcbb2362a9aed27cd0 [id=7ed1fb5bfa8b4c43bcc312e3a8df7ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.046084 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:31.060734 16720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760bad8ccecc47418ccbeb8960c018db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.061218 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:31.062086 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:31.062701 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a6894a4ee5fe4884b5162d8055e766fe (table default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e]). Expected version 2 got 0
I20260923 17:31:31.066046 17753 raft_consensus.cc:2781] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:31.066437 17753 raft_consensus.cc:686] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:31.067078 17753 consensus_queue.cc:237] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.080513 16717 tablet_service.cc:1467] Processing CreateTablet for tablet 43bce0fe5fdf459daeeda19dca2665d2 (DEFAULT_TABLE table=default.Table_37d4870a361e4247a261b101b89d1ffb [id=b1f352b253e74ec9a5d1ffe40cd39ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.081599 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bce0fe5fdf459daeeda19dca2665d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.087673 17146 raft_consensus.cc:348] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.096833 17146 raft_consensus.cc:374] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.096694 16866 tablet_service.cc:1467] Processing CreateTablet for tablet bc14a3192ad9429ea31bb7f1a7674ef7 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.102428 17146 raft_consensus.cc:729] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.107514 17146 consensus_queue.cc:260] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.127516 18167 raft_consensus.cc:482] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.128271 18167 raft_consensus.cc:504] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.130725 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:31.134776 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760bad8ccecc47418ccbeb8960c018db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.098917 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760bad8ccecc47418ccbeb8960c018db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.094884 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 43bce0fe5fdf459daeeda19dca2665d2 (DEFAULT_TABLE table=default.Table_37d4870a361e4247a261b101b89d1ffb [id=b1f352b253e74ec9a5d1ffe40cd39ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.118184 17015 tablet_service.cc:1467] Processing CreateTablet for tablet bc14a3192ad9429ea31bb7f1a7674ef7 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.145731 16385 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:31.161042 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bce0fe5fdf459daeeda19dca2665d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.166184 17144 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:31.171655 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc14a3192ad9429ea31bb7f1a7674ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.177387 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bce0fe5fdf459daeeda19dca2665d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.179698 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc14a3192ad9429ea31bb7f1a7674ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.184965 16870 tablet_service.cc:1467] Processing CreateTablet for tablet b116dab2c1884e02be43afccbb6dd3d7 (DEFAULT_TABLE table=default.Table_044d6aa80d5d44339d4928ad987fd8c5 [id=d88025fb03244d63af0dd965013441d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.185889 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b116dab2c1884e02be43afccbb6dd3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.186685 17008 tablet_service.cc:1467] Processing CreateTablet for tablet b116dab2c1884e02be43afccbb6dd3d7 (DEFAULT_TABLE table=default.Table_044d6aa80d5d44339d4928ad987fd8c5 [id=d88025fb03244d63af0dd965013441d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.189389 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b116dab2c1884e02be43afccbb6dd3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.193370 17027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:31.212098 17027 raft_consensus.cc:2379] T adc1f188f61e40a2ae42fcb716f5619c P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:31.212517 17146 ts_tablet_manager.cc:1428] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.178s user 0.002s sys 0.006s
I20260923 17:31:31.214125 17146 tablet_bootstrap.cc:492] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:31.217213 16712 tablet_service.cc:1467] Processing CreateTablet for tablet b116dab2c1884e02be43afccbb6dd3d7 (DEFAULT_TABLE table=default.Table_044d6aa80d5d44339d4928ad987fd8c5 [id=d88025fb03244d63af0dd965013441d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.218549 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b116dab2c1884e02be43afccbb6dd3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.221557 18201 raft_consensus.cc:482] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.222044 18201 raft_consensus.cc:504] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.224187 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e99ac8c0549b188cf0d872f33d1bf" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:31.227005 18201 leader_election.cc:288] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:31.229285 16713 tablet_service.cc:1467] Processing CreateTablet for tablet bc14a3192ad9429ea31bb7f1a7674ef7 (DEFAULT_TABLE table=default.Table_17a12a23debd4da29e0cc2c01cd73d8e [id=037f88212156498f86750160ee8fc2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:31.242246 16713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc14a3192ad9429ea31bb7f1a7674ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:31.266446 17144 tablet_bootstrap.cc:654] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:31.269106 18132 lock_manager.cc:586] Waited 2 seconds to obtain row lock on key '<redacted>' (key hash 2212082103761160720) tablet 00000000000000000000000000000000 cur holder 0x7b5800031800
I20260923 17:31:31.291093 16387 catalog_manager.cc:5495] T 107a073567314be996d6b8e3dbc47701 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:31.295894 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:31.299597 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:31.310699 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:31.312213 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:31.318065 16387 catalog_manager.cc:5495] T 31a83c7168a64b64bd224ef5a4c6939e P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:31.191267 18167 leader_election.cc:288] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:31.335304 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:31.338984 16387 catalog_manager.cc:5495] T 9f2e99ac8c0549b188cf0d872f33d1bf P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:31.354378 17143 tablet_bootstrap.cc:492] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:31.354880 17143 ts_tablet_manager.cc:1397] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.499s user 0.014s sys 0.016s
I20260923 17:31:31.356253 16387 catalog_manager.cc:5495] T 0e5b5504f7174bf483e9ddc79161079f P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:31.365095 16387 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
W20260923 17:31:31.345625 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 286715us
I20260923 17:31:31.377177 17146 tablet_bootstrap.cc:654] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:31.381642 17143 raft_consensus.cc:348] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.382297 17143 raft_consensus.cc:374] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.390839 16384 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 232) took 4022 ms. Trace:
I20260923 17:31:31.391153 16384 rpcz_store.cc:276] 0923 17:31:27.369643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.962404 (+1592761us) service_pool.cc:227] Handling call
0923 17:31:28.963551 (+ 1147us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:28.964183 (+ 632us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:28.964777 (+ 594us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:28.964981 (+ 204us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:29.074159 (+109178us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:31.390776 (+2316617us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.051104 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.051495 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.051518 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:29.051531 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:29.051791 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.051825 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.051850 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:29.051988 (+ 138us) tablet.cc:699] Row locks acquired
0923 17:31:29.052003 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:29.052108 (+ 105us) write_op.cc:270] Start()
0923 17:31:29.052188 (+ 80us) write_op.cc:276] Timestamp: P: 1790184689052062 usec, L: 0
0923 17:31:29.052205 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.052705 (+ 500us) log.cc:842] Serialized 405 byte log entry
0923 17:31:29.066045 (+ 13340us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.069134 (+ 3089us) write_op.cc:301] APPLY: starting
0923 17:31:29.069182 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.069287 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.069382 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.069555 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.069668 (+ 113us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:29.069706 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:29.069866 (+ 160us) log.cc:842] Serialized 23 byte log entry
0923 17:31:29.070014 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.070070 (+ 56us) write_op.cc:454] Released schema lock
0923 17:31:29.070097 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:29.070250 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.070271 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2951,"apply.run_cpu_time_us":1288,"apply.run_wall_time_us":1288,"mutex_wait_us":85062,"num_ops":1,"prepare.queue_time_us":85149,"prepare.run_cpu_time_us":1790,"prepare.run_wall_time_us":3257,"replication_time_us":13777}]]}
W20260923 17:31:31.407953 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 39) took 1626 ms (client timeout 1000 ms). Trace:
W20260923 17:31:31.409243 16384 rpcz_store.cc:269] 0923 17:31:29.781361 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.407773 (+1626412us) service_pool.cc:214] Skipping call since client already timed out
0923 17:31:31.407882 (+ 109us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260923 17:31:31.413328 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 36) took 1317 ms (client timeout 1000 ms). Trace:
W20260923 17:31:31.413695 16384 rpcz_store.cc:269] 0923 17:31:30.095572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.413177 (+1317605us) service_pool.cc:214] Skipping call since client already timed out
0923 17:31:31.413270 (+ 93us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260923 17:31:31.414850 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 295996us
I20260923 17:31:31.449128 16388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 5c929b8beaaf4c7aa668852b63af2204
I20260923 17:31:31.450418 16388 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 228) took 6365 ms. Trace:
I20260923 17:31:31.457871 16385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e] alter complete (version 1)
I20260923 17:31:31.450691 16388 rpcz_store.cc:276] 0923 17:31:25.084575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.071105 (+2986530us) service_pool.cc:227] Handling call
0923 17:31:28.199389 (+128284us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:28.199871 (+ 482us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:28.200007 (+ 136us) catalog_manager.cc:2761] Locking tablets
0923 17:31:28.200497 (+ 490us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:28.936402 (+735905us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:28.936602 (+ 200us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:31.449024 (+2512422us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:31.450365 (+ 1341us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:28.693997 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:28.694971 (+ 974us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:28.695047 (+ 76us) write_op.cc:435] Acquired schema lock
0923 17:31:28.695091 (+ 44us) tablet.cc:660] Decoding operations
0923 17:31:28.695838 (+ 747us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:28.695909 (+ 71us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:28.695985 (+ 76us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:28.710631 (+ 14646us) tablet.cc:699] Row locks acquired
0923 17:31:28.710648 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:28.710779 (+ 131us) write_op.cc:270] Start()
0923 17:31:28.710896 (+ 117us) write_op.cc:276] Timestamp: P: 1790184688710707 usec, L: 0
0923 17:31:28.710914 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:28.711435 (+ 521us) log.cc:842] Serialized 765 byte log entry
0923 17:31:28.929509 (+218074us) op_driver.cc:444] REPLICATION: finished
0923 17:31:28.931666 (+ 2157us) write_op.cc:301] APPLY: starting
0923 17:31:28.931727 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:28.931839 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:28.932097 (+ 258us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:28.932840 (+ 743us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:28.933154 (+ 314us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:28.933244 (+ 90us) write_op.cc:312] APPLY: finished
0923 17:31:28.933644 (+ 400us) log.cc:842] Serialized 29 byte log entry
0923 17:31:28.933808 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.933873 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:28.933905 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:28.934107 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:28.934129 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1968,"apply.run_cpu_time_us":2704,"apply.run_wall_time_us":2706,"num_ops":2,"prepare.queue_time_us":430282,"prepare.run_cpu_time_us":3627,"prepare.run_wall_time_us":21217,"replication_time_us":218539}]]}
I20260923 17:31:31.474539 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:31.499020 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 54201.7us
I20260923 17:31:31.513887 18152 raft_consensus.cc:482] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.514339 18152 raft_consensus.cc:504] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.516469 18152 leader_election.cc:288] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:31.518858 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:31.521005 16972 leader_election.cc:341] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:31.515161 17144 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:31.522116 17144 ts_tablet_manager.cc:1397] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.356s user 0.009s sys 0.006s
I20260923 17:31:31.454873 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.507143 16867 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.507579 16872 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.507635 16868 tablet_service.cc:1514] Processing DeleteTablet for tablet a60bf118082b4e9991bfeb3ac59eb118 with delete_type TABLET_DATA_DELETED (replaced by tablet 261d1d03290546969a74ca60beb890eb) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.508386 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.509090 16871 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:18 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.513245 16863 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.517562 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:31.523669 16862 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:14 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.524461 16861 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:31.532948 16972 leader_election.cc:341] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:31.533402 16972 leader_election.cc:302] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:31.496170 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:31.534380 18193 raft_consensus.cc:2726] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:31.536890 17143 raft_consensus.cc:729] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.538254 16826 leader_election.cc:302] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:31.539090 18167 raft_consensus.cc:2726] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:31.538952 17143 consensus_queue.cc:260] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.541198 17143 ts_tablet_manager.cc:1428] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.186s user 0.004s sys 0.006s
I20260923 17:31:31.541689 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c115e29673747f8b0712d441ac993fb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:31.545506 17144 raft_consensus.cc:348] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.546908 17144 raft_consensus.cc:374] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.547191 17144 raft_consensus.cc:729] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.556751 17144 consensus_queue.cc:260] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:31.575827 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 37) took 3947 ms (client timeout 1000 ms). Trace:
I20260923 17:31:31.582055 17146 tablet_bootstrap.cc:492] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:31.582541 17146 ts_tablet_manager.cc:1397] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.369s user 0.017s sys 0.005s
I20260923 17:31:31.604765 18222 tablet_replica.cc:330] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
W20260923 17:31:31.610872 16380 rpcz_store.cc:269] 0923 17:31:27.628052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.770892 (+142840us) service_pool.cc:227] Handling call
0923 17:31:31.575748 (+3804856us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21778,"reported_tablets":39}
I20260923 17:31:31.617266 17146 raft_consensus.cc:348] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.623154 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:31.625579 16383 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_b3879296b41f459ab8637f32909c42f6 [id=02fecf5168494501aca594904d9e933e] alter complete (version 2)
I20260923 17:31:31.626415 16893 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:31.636372 17146 raft_consensus.cc:374] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.636683 17146 raft_consensus.cc:729] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.633289 16892 raft_consensus.cc:3037] T 9f2e99ac8c0549b188cf0d872f33d1bf P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:31.644650 17146 consensus_queue.cc:260] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.646574 17146 ts_tablet_manager.cc:1428] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.064s user 0.003s sys 0.003s
I20260923 17:31:31.647034 18219 tablet_replica.cc:330] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:31.657027 17144 ts_tablet_manager.cc:1428] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.133s user 0.004s sys 0.004s
I20260923 17:31:31.674249 18221 tablet_replica.cc:330] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:31.674280 17143 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:31.677421 18221 raft_consensus.cc:2229] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:31.678399 18221 raft_consensus.cc:2258] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:31.682195 18221 ts_tablet_manager.cc:1905] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:31.687076 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:31.692451 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:31.692607 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:31.695029 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:31.695650 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:31.699484 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet df53a9825e5348189485f30b8fdedc63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:23 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:31.708758 18219 raft_consensus.cc:2229] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:31.710330 18219 raft_consensus.cc:2258] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:31.713244 18219 ts_tablet_manager.cc:1905] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:31.746057 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:31.747268 17144 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:31.751753 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (table default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e6d9fad7151a4551ba8f2b0e25525cc7 consensus state has no leader
I20260923 17:31:31.754004 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:31.758392 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:31.761725 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1f188f61e40a2ae42fcb716f5619c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:31.768699 17146 tablet_bootstrap.cc:492] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:31.783704 18214 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.416s user 0.020s sys 0.007s
W20260923 17:31:31.806318 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 33) took 5576 ms (client timeout 1000 ms). Trace:
W20260923 17:31:31.809648 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 34) took 4493 ms (client timeout 1000 ms). Trace:
I20260923 17:31:31.813051 17146 tablet_bootstrap.cc:654] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:31.816102 16381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 41) took 4817 ms (client timeout 1000 ms). Trace:
W20260923 17:31:31.810268 16386 rpcz_store.cc:269] 0923 17:31:27.316148 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.764522 (+448374us) service_pool.cc:227] Handling call
0923 17:31:30.003028 (+2238506us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:30.003056 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:31.809585 (+1806529us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.984833 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.985565 (+ 732us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.985627 (+ 62us) write_op.cc:435] Acquired schema lock
0923 17:31:29.985680 (+ 53us) tablet.cc:660] Decoding operations
0923 17:31:29.986117 (+ 437us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.986161 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.986191 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:29.986366 (+ 175us) tablet.cc:699] Row locks acquired
0923 17:31:29.986424 (+ 58us) write_op.cc:260] PREPARE: finished
0923 17:31:29.986532 (+ 108us) write_op.cc:270] Start()
0923 17:31:29.986612 (+ 80us) write_op.cc:276] Timestamp: P: 1790184689986486 usec, L: 0
0923 17:31:29.986629 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.987197 (+ 568us) log.cc:842] Serialized 536 byte log entry
0923 17:31:29.995929 (+ 8732us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.996266 (+ 337us) write_op.cc:301] APPLY: starting
0923 17:31:29.996321 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.996421 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.996507 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.996680 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.996831 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:29.996918 (+ 87us) write_op.cc:312] APPLY: finished
0923 17:31:29.997237 (+ 319us) log.cc:842] Serialized 23 byte log entry
0923 17:31:29.997367 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.997443 (+ 76us) write_op.cc:454] Released schema lock
0923 17:31:29.997477 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":96860,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":184,"apply.run_cpu_time_us":1399,"apply.run_wall_time_us":1405,"num_ops":1,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":2765,"prepare.run_wall_time_us":2766,"raft.queue_time_us":830,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":528,"replication_time_us":9234}]]}
I20260923 17:31:31.830302 18190 raft_consensus.cc:482] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.831039 18190 raft_consensus.cc:504] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:31.827679 16381 rpcz_store.cc:269] 0923 17:31:26.999118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.280337 (+281219us) service_pool.cc:227] Handling call
0923 17:31:29.802908 (+2522571us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.802930 (+ 22us) write_op.cc:454] Released schema lock
0923 17:31:31.816041 (+2013111us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.786207 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.786619 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.786643 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:29.786664 (+ 21us) tablet.cc:660] Decoding operations
0923 17:31:29.786966 (+ 302us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.787005 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.787027 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:29.787186 (+ 159us) tablet.cc:699] Row locks acquired
0923 17:31:29.787202 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:29.787307 (+ 105us) write_op.cc:270] Start()
0923 17:31:29.787389 (+ 82us) write_op.cc:276] Timestamp: P: 1790184689787224 usec, L: 0
0923 17:31:29.787408 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.787893 (+ 485us) log.cc:842] Serialized 536 byte log entry
0923 17:31:29.797223 (+ 9330us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.797483 (+ 260us) write_op.cc:301] APPLY: starting
0923 17:31:29.797534 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.797617 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.797721 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.797909 (+ 188us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.798032 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:29.798072 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:29.798227 (+ 155us) log.cc:842] Serialized 23 byte log entry
0923 17:31:29.798349 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.798406 (+ 57us) write_op.cc:454] Released schema lock
0923 17:31:29.798434 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":60507,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":1104,"apply.run_wall_time_us":1103,"num_ops":1,"prepare.queue_time_us":1543,"prepare.run_cpu_time_us":1949,"prepare.run_wall_time_us":1949,"raft.queue_time_us":1173,"raft.run_cpu_time_us":500,"raft.run_wall_time_us":500,"replication_time_us":8772}]]}
I20260923 17:31:31.834812 18190 leader_election.cc:288] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:31.836985 18187 raft_consensus.cc:482] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.837523 18187 raft_consensus.cc:504] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:31.806960 16382 rpcz_store.cc:269] 0923 17:31:26.230333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:27.172383 (+942050us) service_pool.cc:227] Handling call
0923 17:31:31.806244 (+4633861us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.368246 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.368730 (+ 484us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.368787 (+ 57us) write_op.cc:435] Acquired schema lock
0923 17:31:29.368814 (+ 27us) tablet.cc:660] Decoding operations
0923 17:31:29.369231 (+ 417us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.369278 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.369320 (+ 42us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:29.369535 (+ 215us) tablet.cc:699] Row locks acquired
0923 17:31:29.369550 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:29.369690 (+ 140us) write_op.cc:270] Start()
0923 17:31:29.369808 (+ 118us) write_op.cc:276] Timestamp: P: 1790184689369623 usec, L: 0
0923 17:31:29.369829 (+ 21us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.370552 (+ 723us) log.cc:842] Serialized 536 byte log entry
0923 17:31:29.380867 (+ 10315us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.381248 (+ 381us) write_op.cc:301] APPLY: starting
0923 17:31:29.381302 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.381394 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.381489 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.381737 (+ 248us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.381953 (+ 216us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:29.382025 (+ 72us) write_op.cc:312] APPLY: finished
0923 17:31:29.382332 (+ 307us) log.cc:842] Serialized 23 byte log entry
0923 17:31:29.382612 (+ 280us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.382712 (+ 100us) write_op.cc:454] Released schema lock
0923 17:31:29.382782 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:29.383172 (+ 390us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.383199 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":243150,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":198,"apply.run_cpu_time_us":2134,"apply.run_wall_time_us":3843,"num_ops":1,"prepare.queue_time_us":100316,"prepare.run_cpu_time_us":2697,"prepare.run_wall_time_us":2697,"raft.queue_time_us":102,"raft.run_cpu_time_us":566,"raft.run_wall_time_us":566,"replication_time_us":10951,"spinlock_wait_cycles":87936}]]}
I20260923 17:31:31.841226 18187 leader_election.cc:288] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:31.841460 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3759087d3e8a4ccd91f2b0436ffdc062"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:31.843335 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3759087d3e8a4ccd91f2b0436ffdc062 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:31.850941 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_8682ef6c78e7491cbc609893b5f383bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:31.859062 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8682ef6c78e7491cbc609893b5f383bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:31.847656 16382 master_service.cc:946] ReplaceTablet: received request to replace tablet 456273cf57314cb8a05c8e765543913f from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:31.747869 17143 tablet_bootstrap.cc:654] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:31.836834 16938 heartbeater.cc:347] Connected to a master server at 127.15.246.188:46075
I20260923 17:31:31.856760 16886 raft_consensus.cc:3037] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:31.877686 16678 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 7c115e29673747f8b0712d441ac993fb
I20260923 17:31:31.879179 18204 consensus_queue.cc:1035] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:31.880538 17146 tablet_bootstrap.cc:492] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:31.881093 18204 consensus_queue.cc:1035] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:31.883654 18230 log.cc:924] Time spent T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260923 17:31:31.892468 17146 ts_tablet_manager.cc:1397] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.124s user 0.010s sys 0.005s
I20260923 17:31:31.895599 17146 raft_consensus.cc:348] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.896382 17146 raft_consensus.cc:374] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.896677 17146 raft_consensus.cc:729] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.898485 17143 tablet_bootstrap.cc:492] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:31.898918 17143 ts_tablet_manager.cc:1397] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.225s user 0.014s sys 0.005s
I20260923 17:31:31.901144 17143 raft_consensus.cc:348] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.901880 17143 raft_consensus.cc:374] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:31.902148 17143 raft_consensus.cc:729] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:31.902853 17143 consensus_queue.cc:260] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.904847 17143 ts_tablet_manager.cc:1428] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260923 17:31:31.909075 17143 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:31.927173 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:31.928906 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 3 got 2
I20260923 17:31:31.929870 17146 consensus_queue.cc:260] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:31.931205 17144 tablet_bootstrap.cc:654] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:31.958732 17143 tablet_bootstrap.cc:654] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:31.959143 17146 ts_tablet_manager.cc:1428] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.066s user 0.005s sys 0.000s
I20260923 17:31:31.960040 17146 tablet_bootstrap.cc:492] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:31.969134 18219 ts_tablet_manager.cc:1918] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:31.969589 18219 log.cc:1196] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/5c929b8beaaf4c7aa668852b63af2204
I20260923 17:31:31.970391 18219 ts_tablet_manager.cc:1939] T 5c929b8beaaf4c7aa668852b63af2204 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:31.971679 16386 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 237) took 2806 ms. Trace:
I20260923 17:31:31.972569 16381 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 238) took 2675 ms. Trace:
I20260923 17:31:31.974413 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:31.973129 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 5c929b8beaaf4c7aa668852b63af2204 (table default.Table_8be9935ea009465aaaae2c81f5fa6783 [id=aaedb0c6cff242f2acaf82970a6c0e30]) successfully deleted
I20260923 17:31:31.975366 16388 catalog_manager.cc:5495] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } }
I20260923 17:31:31.976548 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:31.977455 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:31.973492 16386 rpcz_store.cc:276] 0923 17:31:29.164822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.834966 (+2670144us) service_pool.cc:227] Handling call
0923 17:31:31.859522 (+ 24556us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:31.860880 (+ 1358us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:31.964315 (+103435us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:31.964605 (+ 290us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:31.971615 (+ 7010us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:31.903109 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.903521 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.903545 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:31.903559 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:31.903920 (+ 361us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.903955 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.903978 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:31.904188 (+ 210us) tablet.cc:699] Row locks acquired
0923 17:31:31.904200 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:31.904380 (+ 180us) write_op.cc:270] Start()
0923 17:31:31.904467 (+ 87us) write_op.cc:276] Timestamp: P: 1790184691904281 usec, L: 0
0923 17:31:31.904485 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.905074 (+ 589us) log.cc:842] Serialized 433 byte log entry
0923 17:31:31.957381 (+ 52307us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.958326 (+ 945us) write_op.cc:301] APPLY: starting
0923 17:31:31.958377 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.958467 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.958552 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.958923 (+ 371us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.959044 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:31.959081 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:31.959243 (+ 162us) log.cc:842] Serialized 29 byte log entry
0923 17:31:31.959367 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.959427 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:31.959454 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:31.959600 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.959621 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":809,"apply.run_cpu_time_us":1434,"apply.run_wall_time_us":1434,"mutex_wait_us":40998,"num_ops":2,"prepare.queue_time_us":41214,"prepare.run_cpu_time_us":2030,"prepare.run_wall_time_us":3110,"replication_time_us":52823}]]}
I20260923 17:31:31.974656 16381 rpcz_store.cc:276] 0923 17:31:29.297451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.840899 (+2543448us) service_pool.cc:227] Handling call
0923 17:31:31.855455 (+ 14556us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:31.856893 (+ 1438us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:31.972018 (+115125us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:31.972313 (+ 295us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:31.972521 (+ 208us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:31.885577 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.886075 (+ 498us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.886102 (+ 27us) write_op.cc:435] Acquired schema lock
0923 17:31:31.886115 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:31.886421 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.886467 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.886492 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:31.886702 (+ 210us) tablet.cc:699] Row locks acquired
0923 17:31:31.886716 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:31.886853 (+ 137us) write_op.cc:270] Start()
0923 17:31:31.886938 (+ 85us) write_op.cc:276] Timestamp: P: 1790184691886756 usec, L: 0
0923 17:31:31.886954 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.887480 (+ 526us) log.cc:842] Serialized 433 byte log entry
0923 17:31:31.943960 (+ 56480us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.944295 (+ 335us) write_op.cc:301] APPLY: starting
0923 17:31:31.944344 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.944436 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.944533 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.944932 (+ 399us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.945060 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:31.945097 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:31.945244 (+ 147us) log.cc:842] Serialized 29 byte log entry
0923 17:31:31.945372 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.945432 (+ 60us) write_op.cc:454] Released schema lock
0923 17:31:31.945460 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:31.945610 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.945633 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":178,"apply.run_cpu_time_us":1478,"apply.run_wall_time_us":1477,"num_ops":2,"prepare.queue_time_us":27489,"prepare.run_cpu_time_us":2178,"prepare.run_wall_time_us":2178,"raft.queue_time_us":1144,"raft.run_cpu_time_us":621,"raft.run_wall_time_us":621,"replication_time_us":56930}]]}
I20260923 17:31:31.982996 16381 master_service.cc:946] ReplaceTablet: received request to replace tablet 1feebaa76bcd4ff5b707814191cb3341 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:31.981508 16386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_219bd5224fe64024867dec9b8bef6824" } modify_external_catalogs: true
I20260923 17:31:31.983628 16386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_219bd5224fe64024867dec9b8bef6824" } modify_external_catalogs: true
I20260923 17:31:31.994060 18167 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:31.994827 18167 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:31.998798 18167 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.004879 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.005726 17025 raft_consensus.cc:2454] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 1.
I20260923 17:31:32.007050 16827 leader_election.cc:302] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:32.007992 18167 raft_consensus.cc:2781] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 17:31:32.008385 18167 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:32.008674 18167 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 17:31:32.017270 18167 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.018169 17146 tablet_bootstrap.cc:654] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.021119 18151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:32.021638 18151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:32.026194 18116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbd0d3324414f26ab5d34c375921ef2. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:32.026894 18116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:32.031001 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:32.027984 18215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:32.031628 18215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:32.031636 17025 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Advancing to term 2
I20260923 17:31:32.034935 18151 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:32.044013 18167 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 election: Requested vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.052815 17936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38696 (request call id 0) took 3471 ms. Trace:
I20260923 17:31:32.044598 16826 leader_election.cc:302] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:32.064594 18167 raft_consensus.cc:2726] T adc1f188f61e40a2ae42fcb716f5619c P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.073153 18236 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 17:31:32.075064 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:32.075860 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:32.076848 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
W20260923 17:31:32.078346 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 194823us
W20260923 17:31:32.125135 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 52109.5us
W20260923 17:31:32.125767 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 44146.5us
I20260923 17:31:32.135565 17143 tablet_bootstrap.cc:492] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:32.143128 17143 ts_tablet_manager.cc:1397] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.234s user 0.006s sys 0.004s
I20260923 17:31:32.146045 17143 raft_consensus.cc:348] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.101738 17025 raft_consensus.cc:2454] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36e79694e154f84a46980719217d7f2 in term 2.
I20260923 17:31:32.151335 17146 tablet_bootstrap.cc:492] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:32.151707 17146 ts_tablet_manager.cc:1397] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.192s user 0.009s sys 0.003s
I20260923 17:31:32.151799 18219 tablet_replica.cc:330] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
W20260923 17:31:32.153240 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd because the tablet was not found. No further retry: Not found: Tablet not found: 19ebfb3cfd3d49b39c7cf453199ac8bd
W20260923 17:31:32.153927 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 261d1d03290546969a74ca60beb890eb because the tablet was not found. No further retry: Not found: Tablet not found: 261d1d03290546969a74ca60beb890eb
I20260923 17:31:32.150629 16827 leader_election.cc:302] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:32.154770 18167 raft_consensus.cc:2781] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 17:31:32.155243 18167 raft_consensus.cc:686] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 2 LEADER]: Becoming Leader. State: Replica: c36e79694e154f84a46980719217d7f2, State: Running, Role: LEADER
I20260923 17:31:32.155944 17146 raft_consensus.cc:348] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.156620 17146 raft_consensus.cc:374] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.156039 18167 consensus_queue.cc:237] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.156853 17146 raft_consensus.cc:729] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.157461 17146 consensus_queue.cc:260] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.170364 18222 raft_consensus.cc:2229] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:32.172694 17143 raft_consensus.cc:374] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.173038 17143 raft_consensus.cc:729] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.160831 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:32.173218 18219 raft_consensus.cc:2229] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:32.174077 18219 raft_consensus.cc:2258] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:32.175773 16386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 240) took 2557 ms. Trace:
I20260923 17:31:32.176075 16386 rpcz_store.cc:276] 0923 17:31:29.618519 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.981225 (+2362706us) service_pool.cc:227] Handling call
0923 17:31:31.984076 (+ 2851us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:31.984581 (+ 505us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:31.984684 (+ 103us) catalog_manager.cc:2761] Locking tablets
0923 17:31:32.105974 (+121290us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:32.160448 (+ 54474us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:32.160649 (+ 201us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:32.160745 (+ 96us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:32.175713 (+ 14968us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.116601 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.117022 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.117052 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:32.117065 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:32.117810 (+ 745us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.117858 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.117887 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.118288 (+ 401us) tablet.cc:699] Row locks acquired
0923 17:31:32.118302 (+ 14us) write_op.cc:260] PREPARE: finished
0923 17:31:32.118436 (+ 134us) write_op.cc:270] Start()
0923 17:31:32.118521 (+ 85us) write_op.cc:276] Timestamp: P: 1790184692118326 usec, L: 0
0923 17:31:32.118540 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.119049 (+ 509us) log.cc:842] Serialized 734 byte log entry
0923 17:31:32.158324 (+ 39275us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.159014 (+ 690us) write_op.cc:301] APPLY: starting
0923 17:31:32.159072 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.159172 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.159263 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.159511 (+ 248us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.159652 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:32.159692 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:32.159857 (+ 165us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.159994 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.160053 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:32.160082 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:32.160264 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.160285 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":532,"apply.run_cpu_time_us":1458,"apply.run_wall_time_us":1458,"num_ops":2,"prepare.queue_time_us":9043,"prepare.run_cpu_time_us":2784,"prepare.run_wall_time_us":21384,"raft.queue_time_us":16759,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":716,"replication_time_us":39692}]]}
I20260923 17:31:32.181289 16386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_9394ecdeb8cf4220b38b8bf6b6a51bc4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:32.182237 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:31 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:32.182698 16386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9394ecdeb8cf4220b38b8bf6b6a51bc4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:32.185652 16382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 456273cf57314cb8a05c8e765543913f
I20260923 17:31:32.186836 16382 catalog_manager.cc:6565] ReplaceTablet: tablet 456273cf57314cb8a05c8e765543913f deleted and replaced by tablet 3fc89f029c1c4d8fb1b192017cade4f7
I20260923 17:31:32.187158 16382 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 239) took 2701 ms. Trace:
I20260923 17:31:32.186980 18221 ts_tablet_manager.cc:1918] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:32.187655 18221 log.cc:1196] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/5c929b8beaaf4c7aa668852b63af2204
I20260923 17:31:32.187417 16382 rpcz_store.cc:276] 0923 17:31:29.485926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.847526 (+2361600us) service_pool.cc:227] Handling call
0923 17:31:32.187110 (+339584us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.138110 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.138609 (+ 499us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.138661 (+ 52us) write_op.cc:435] Acquired schema lock
0923 17:31:32.138682 (+ 21us) tablet.cc:660] Decoding operations
0923 17:31:32.139072 (+ 390us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.139121 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.139151 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.139462 (+ 311us) tablet.cc:699] Row locks acquired
0923 17:31:32.139479 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:32.139595 (+ 116us) write_op.cc:270] Start()
0923 17:31:32.139682 (+ 87us) write_op.cc:276] Timestamp: P: 1790184692139481 usec, L: 0
0923 17:31:32.139697 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.147056 (+ 7359us) log.cc:842] Serialized 665 byte log entry
0923 17:31:32.182078 (+ 35022us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.182632 (+ 554us) write_op.cc:301] APPLY: starting
0923 17:31:32.182685 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.183799 (+ 1114us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.183884 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.184214 (+ 330us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.184450 (+ 236us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:32.184488 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:32.184656 (+ 168us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.184795 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.184860 (+ 65us) write_op.cc:454] Released schema lock
0923 17:31:32.184889 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:32.185040 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.185063 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":412,"apply.run_cpu_time_us":1606,"apply.run_wall_time_us":2594,"num_ops":2,"prepare.queue_time_us":28983,"prepare.run_cpu_time_us":2485,"prepare.run_wall_time_us":14924,"replication_time_us":42309}]]}
I20260923 17:31:32.167917 17146 ts_tablet_manager.cc:1428] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260923 17:31:32.191476 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_5f3593d4362a4d0d8797c69b90733b9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:32.192631 17146 tablet_bootstrap.cc:492] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:32.192950 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f3593d4362a4d0d8797c69b90733b9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:32.194039 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 67570.9us
I20260923 17:31:32.196822 18222 raft_consensus.cc:2258] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:32.198891 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 270 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:32.201745 18222 ts_tablet_manager.cc:1905] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:32.202900 18241 raft_consensus.cc:482] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.208863 16866 tablet_service.cc:1514] Processing DeleteTablet for tablet 456273cf57314cb8a05c8e765543913f with delete_type TABLET_DATA_DELETED (replaced by tablet 3fc89f029c1c4d8fb1b192017cade4f7) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:32.211054 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 270 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:32.212114 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:31 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:32.212770 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 456273cf57314cb8a05c8e765543913f with delete_type TABLET_DATA_DELETED (replaced by tablet 3fc89f029c1c4d8fb1b192017cade4f7) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:32.214349 18219 ts_tablet_manager.cc:1905] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:32.205080 17143 consensus_queue.cc:260] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.207866 18241 raft_consensus.cc:504] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.224946 17884 consensus_queue.cc:1035] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:32.228863 16827 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:32.231379 17146 tablet_bootstrap.cc:654] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.233297 17860 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:32.234865 18151 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260923 17:31:32.243104 17143 ts_tablet_manager.cc:1428] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.087s user 0.005s sys 0.000s
I20260923 17:31:32.246713 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.247419 17025 raft_consensus.cc:2454] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:32.249658 17040 raft_consensus.cc:1226] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 2 FOLLOWER]: Rejecting Update request from peer be93fda0843b44f5abc49e6f35c69217 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260923 17:31:32.249752 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.250378 17025 raft_consensus.cc:2454] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:32.263087 16382 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 243) took 2524 ms. Trace:
I20260923 17:31:32.263419 16382 rpcz_store.cc:276] 0923 17:31:29.738355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.190944 (+2452589us) service_pool.cc:227] Handling call
0923 17:31:32.193299 (+ 2355us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:32.194628 (+ 1329us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:32.262497 (+ 67869us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:32.262777 (+ 280us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:32.263025 (+ 248us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.243950 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.244380 (+ 430us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.244403 (+ 23us) write_op.cc:435] Acquired schema lock
0923 17:31:32.244419 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:32.244702 (+ 283us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.244744 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.244769 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.244962 (+ 193us) tablet.cc:699] Row locks acquired
0923 17:31:32.244987 (+ 25us) write_op.cc:260] PREPARE: finished
0923 17:31:32.245107 (+ 120us) write_op.cc:270] Start()
0923 17:31:32.245181 (+ 74us) write_op.cc:276] Timestamp: P: 1790184692244991 usec, L: 0
0923 17:31:32.245198 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.245685 (+ 487us) log.cc:842] Serialized 433 byte log entry
0923 17:31:32.258285 (+ 12600us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.258970 (+ 685us) write_op.cc:301] APPLY: starting
0923 17:31:32.259037 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.259135 (+ 98us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.259225 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.259606 (+ 381us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.259735 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:32.259776 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:32.259937 (+ 161us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.260073 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.260134 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:32.260162 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:32.260334 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.260360 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":554,"apply.run_cpu_time_us":1536,"apply.run_wall_time_us":1535,"num_ops":2,"prepare.queue_time_us":48239,"prepare.run_cpu_time_us":2038,"prepare.run_wall_time_us":2942,"raft.queue_time_us":83,"raft.run_cpu_time_us":588,"raft.run_wall_time_us":589,"replication_time_us":13013,"spinlock_wait_cycles":25472}]]}
I20260923 17:31:32.274163 18250 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:32.278771 16386 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 242) took 2588 ms. Trace:
I20260923 17:31:32.287458 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:31 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:32.288928 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 456273cf57314cb8a05c8e765543913f with delete_type TABLET_DATA_DELETED (replaced by tablet 3fc89f029c1c4d8fb1b192017cade4f7) from {username='slave'} at 127.0.0.1:42676
W20260923 17:31:32.297976 16677 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.302328 16677 leader_election.cc:302] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217; no voters: c36e79694e154f84a46980719217d7f2, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:32.306494 17144 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:32.307168 18012 raft_consensus.cc:2726] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.310714 17144 ts_tablet_manager.cc:1397] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.564s user 0.008s sys 0.012s
I20260923 17:31:32.317380 18012 raft_consensus.cc:482] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.317936 18012 raft_consensus.cc:504] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.320003 18012 leader_election.cc:288] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:32.321527 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:32.323153 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 3 got 2
I20260923 17:31:32.331696 17904 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" instance_seqno: 1790184667254068) as {username='slave'} at 127.15.246.131:50773; Asking this server to re-register.
I20260923 17:31:32.336050 18243 consensus_queue.cc:1035] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.279134 16386 rpcz_store.cc:276] 0923 17:31:29.690370 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.179356 (+2488986us) service_pool.cc:227] Handling call
0923 17:31:32.183059 (+ 3703us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:32.247329 (+ 64270us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:32.277984 (+ 30655us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.278010 (+ 26us) write_op.cc:454] Released schema lock
0923 17:31:32.278198 (+ 188us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:32.278478 (+ 280us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:32.278713 (+ 235us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.248475 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.253929 (+ 5454us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.253954 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:32.253965 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:32.254278 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.254317 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.254340 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.254529 (+ 189us) tablet.cc:699] Row locks acquired
0923 17:31:32.254546 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:32.254661 (+ 115us) write_op.cc:270] Start()
0923 17:31:32.254743 (+ 82us) write_op.cc:276] Timestamp: P: 1790184692254542 usec, L: 0
0923 17:31:32.254756 (+ 13us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.255261 (+ 505us) log.cc:842] Serialized 433 byte log entry
0923 17:31:32.265055 (+ 9794us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.265994 (+ 939us) write_op.cc:301] APPLY: starting
0923 17:31:32.266050 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.266147 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.266229 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.266591 (+ 362us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.266714 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:32.266852 (+ 138us) write_op.cc:312] APPLY: finished
0923 17:31:32.267048 (+ 196us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.267193 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.267262 (+ 69us) write_op.cc:454] Released schema lock
0923 17:31:32.267370 (+ 108us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":799,"apply.run_cpu_time_us":1517,"apply.run_wall_time_us":1575,"num_ops":2,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":1925,"prepare.run_wall_time_us":13269,"replication_time_us":10230}]]}
I20260923 17:31:32.348649 17086 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:32.350345 16716 tablet_service.cc:1467] Processing CreateTablet for tablet f4205f6a25f24f1ca0a7d112d115f8b8 (DEFAULT_TABLE table=default.Table_8682ef6c78e7491cbc609893b5f383bb [id=be40fc3d508c42b3874d9d4e6594c53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:32.351884 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4205f6a25f24f1ca0a7d112d115f8b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:32.361367 17144 raft_consensus.cc:348] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.362262 17144 raft_consensus.cc:374] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.362551 17144 raft_consensus.cc:729] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.363215 17144 consensus_queue.cc:260] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.368826 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 1feebaa76bcd4ff5b707814191cb3341
I20260923 17:31:32.368984 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_fb7c50f5b9da47e98737dbf4f90005eb" } new_table_name: "default.Table_04688becafca4883b1c56cbe6ee8132d" modify_external_catalogs: true
I20260923 17:31:32.370051 16381 catalog_manager.cc:6565] ReplaceTablet: tablet 1feebaa76bcd4ff5b707814191cb3341 deleted and replaced by tablet 5c83942d30a543d692641024253078cc
I20260923 17:31:32.370343 16381 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 241) took 2702 ms. Trace:
I20260923 17:31:32.370587 16381 rpcz_store.cc:276] 0923 17:31:29.668033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.982884 (+2314851us) service_pool.cc:227] Handling call
0923 17:31:32.370297 (+387413us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.299901 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.300372 (+ 471us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.300400 (+ 28us) write_op.cc:435] Acquired schema lock
0923 17:31:32.300415 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:32.300731 (+ 316us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.300769 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.300808 (+ 39us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.301006 (+ 198us) tablet.cc:699] Row locks acquired
0923 17:31:32.301022 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:32.301137 (+ 115us) write_op.cc:270] Start()
0923 17:31:32.301212 (+ 75us) write_op.cc:276] Timestamp: P: 1790184692301016 usec, L: 0
0923 17:31:32.301230 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.301725 (+ 495us) log.cc:842] Serialized 631 byte log entry
0923 17:31:32.365629 (+ 63904us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b1800027400. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f91295008b1 00007f912950d661 00007f9127a6d8a8 00007f9127a6fa78 00007f9127a6fa28 00007f9127a6f9a8 00007f9127a6f963 00007f9127a6ec4c 00007f9129592206 00007f912958a808 00007f912761b170
0923 17:31:32.365666 (+ 37us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.366311 (+ 645us) write_op.cc:301] APPLY: starting
0923 17:31:32.366361 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.366463 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.366482 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.366801 (+ 319us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.367003 (+ 202us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:32.367045 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:32.367214 (+ 169us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.367349 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.367415 (+ 66us) write_op.cc:454] Released schema lock
0923 17:31:32.367445 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:32.367599 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.367622 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":505,"apply.run_cpu_time_us":1483,"apply.run_wall_time_us":1483,"num_ops":2,"prepare.queue_time_us":14521,"prepare.run_cpu_time_us":2212,"prepare.run_wall_time_us":40578,"raft.queue_time_us":91,"raft.run_cpu_time_us":625,"raft.run_wall_time_us":626,"replication_time_us":42198,"spinlock_wait_cycles":47676672}]]}
I20260923 17:31:32.375558 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c83942d30a543d692641024253078cc) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:32.380501 16381 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_421fb28f3b474d0899b8e4fc413b0a65" } new_table_name: "default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9" modify_external_catalogs: true
I20260923 17:31:32.382341 16386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_206108d7a4384c5e8a3201f96c3cb705" } modify_external_catalogs: true
I20260923 17:31:32.383042 16386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_206108d7a4384c5e8a3201f96c3cb705" } modify_external_catalogs: true
I20260923 17:31:32.391037 16866 tablet_service.cc:1467] Processing CreateTablet for tablet f4205f6a25f24f1ca0a7d112d115f8b8 (DEFAULT_TABLE table=default.Table_8682ef6c78e7491cbc609893b5f383bb [id=be40fc3d508c42b3874d9d4e6594c53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:32.392071 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4205f6a25f24f1ca0a7d112d115f8b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:32.394254 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 48720.9us
I20260923 17:31:32.395270 17144 ts_tablet_manager.cc:1428] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.080s user 0.004s sys 0.002s
I20260923 17:31:32.403247 17144 tablet_bootstrap.cc:492] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:32.405488 16677 consensus_peers.cc:487] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:32.372649 18152 raft_consensus.cc:482] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.407933 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c83942d30a543d692641024253078cc) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:32.413472 17010 tablet_service.cc:1467] Processing CreateTablet for tablet f4205f6a25f24f1ca0a7d112d115f8b8 (DEFAULT_TABLE table=default.Table_8682ef6c78e7491cbc609893b5f383bb [id=be40fc3d508c42b3874d9d4e6594c53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:32.414685 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4205f6a25f24f1ca0a7d112d115f8b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:32.426735 17144 tablet_bootstrap.cc:654] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.426394 18245 consensus_queue.cc:1035] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.435163 17143 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:32.416903 18152 raft_consensus.cc:504] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.442111 18152 leader_election.cc:288] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.324057 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]). Expected version 3 got 0
I20260923 17:31:32.431728 18250 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42676 (request call id 270) took 1030 ms. Trace:
I20260923 17:31:32.443292 16384 catalog_manager.cc:5495] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:32.444437 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
W20260923 17:31:32.444764 16677 consensus_peers.cc:487] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:32.445713 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:32.453217 16740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:32.455047 16972 leader_election.cc:341] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.456903 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:32.459017 16972 leader_election.cc:341] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.459374 16972 leader_election.cc:302] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:32.464764 18152 raft_consensus.cc:2726] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.478776 18222 ts_tablet_manager.cc:1918] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:32.485180 18219 ts_tablet_manager.cc:1918] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:32.485661 18219 log.cc:1196] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/8e7c8e360d954ce78913e78c8662c341
I20260923 17:31:32.475063 18254 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:32.486507 18219 ts_tablet_manager.cc:1939] T 8e7c8e360d954ce78913e78c8662c341 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:32.487244 18222 log.cc:1196] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/5c929b8beaaf4c7aa668852b63af2204
W20260923 17:31:32.480952 16677 consensus_peers.cc:487] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:32.493484 16382 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 070d38df3f0f43edab3826835c9503ce consensus state has no leader
I20260923 17:31:32.494141 16382 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 244) took 2694 ms. Trace:
I20260923 17:31:32.494525 16382 rpcz_store.cc:276] 0923 17:31:29.800037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.335087 (+2535050us) service_pool.cc:227] Handling call
0923 17:31:32.368901 (+ 33814us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f9127617f2a 00007f91294e3d6c 00007f91295df7d6 00007f9127125634 00007f91271255c2 00007f91271254f1 00007f912712546c 00007f9127124722 00007f91253be94c 00007f91253bdd86
0923 17:31:32.369636 (+ 735us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:32.385057 (+ 15421us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:32.385593 (+ 536us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:32.385780 (+ 187us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:32.429643 (+ 43863us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.429671 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:32.429929 (+ 258us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:32.494088 (+ 64159us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.386845 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.387235 (+ 390us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.387264 (+ 29us) write_op.cc:435] Acquired schema lock
0923 17:31:32.387277 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:32.387575 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.387627 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.387653 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:32.387815 (+ 162us) tablet.cc:699] Row locks acquired
0923 17:31:32.387828 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:32.387940 (+ 112us) write_op.cc:270] Start()
0923 17:31:32.388029 (+ 89us) write_op.cc:276] Timestamp: P: 1790184692387812 usec, L: 0
0923 17:31:32.388045 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.388622 (+ 577us) log.cc:842] Serialized 405 byte log entry
0923 17:31:32.396972 (+ 8350us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.398201 (+ 1229us) write_op.cc:301] APPLY: starting
0923 17:31:32.398268 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.398371 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.398465 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.398626 (+ 161us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.398773 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:32.398815 (+ 42us) write_op.cc:312] APPLY: finished
0923 17:31:32.399011 (+ 196us) log.cc:842] Serialized 23 byte log entry
0923 17:31:32.399182 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.399252 (+ 70us) write_op.cc:454] Released schema lock
0923 17:31:32.399291 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":75950464,"child_traces":[["op",{"apply.queue_time_us":1064,"apply.run_cpu_time_us":1307,"apply.run_wall_time_us":1307,"num_ops":1,"prepare.queue_time_us":99,"prepare.run_cpu_time_us":2002,"prepare.run_wall_time_us":2798,"raft.queue_time_us":75,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":603,"replication_time_us":8844}]]}
I20260923 17:31:32.498373 18222 ts_tablet_manager.cc:1939] T 5c929b8beaaf4c7aa668852b63af2204 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:32.503832 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 271) took 1017 ms. Trace:
I20260923 17:31:32.496203 17904 ts_manager.cc:194] Registered new tserver with Master: e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:32.506199 18222 rpcz_store.cc:276] 0923 17:31:31.486432 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.496008 (+ 9576us) service_pool.cc:227] Handling call
0923 17:31:31.604500 (+108492us) ts_tablet_manager.cc:1145] Deleting tablet 5c929b8beaaf4c7aa668852b63af2204
0923 17:31:31.604581 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:32.371315 (+766734us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:32.478481 (+107166us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:32.503750 (+ 25269us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":584081,"tablet-delete.queue_time_us":70389,"thread_start_us":134700,"threads_started":1}
I20260923 17:31:32.536659 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.537393 17025 raft_consensus.cc:2454] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:32.539275 17146 tablet_bootstrap.cc:492] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:32.539722 17146 ts_tablet_manager.cc:1397] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.347s user 0.009s sys 0.004s
I20260923 17:31:32.540530 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 8e7c8e360d954ce78913e78c8662c341 (table default.Table_384fc4545b2d47ab8cb690f20bcc27d3 [id=5a518abdcc9a44689b8f2074c9e54ef3]) successfully deleted
I20260923 17:31:32.541604 18219 tablet_replica.cc:330] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:32.542685 18219 raft_consensus.cc:2229] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:32.543280 18219 raft_consensus.cc:2258] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:32.545809 18219 ts_tablet_manager.cc:1905] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:32.560195 18222 tablet_replica.cc:330] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:32.560189 16869 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c83942d30a543d692641024253078cc) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:32.443634 18250 rpcz_store.cc:276] 0923 17:31:31.401077 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.401351 (+ 274us) service_pool.cc:227] Handling call
0923 17:31:32.431625 (+1030274us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:31.404429 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0923 17:31:31.420705 (+ 16276us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0923 17:31:31.420739 (+ 34us) alter_schema_op.cc:65] Acquired schema lock
0923 17:31:31.423030 (+ 2291us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0923 17:31:31.423165 (+ 135us) alter_schema_op.cc:126] START. Timestamp: 1790184691423077
0923 17:31:31.423188 (+ 23us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.423852 (+ 664us) log.cc:842] Serialized 173 byte log entry
0923 17:31:32.227540 (+803688us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.269516 (+ 41976us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0923 17:31:32.276287 (+ 6771us) log.cc:842] Serialized 5 byte log entry
0923 17:31:32.379120 (+102833us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:32.388545 (+ 9425us) log.cc:842] Serialized 14 byte log entry
0923 17:31:32.431466 (+ 42921us) alter_schema_op.cc:71] Released schema lock
0923 17:31:32.431494 (+ 28us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":41669,"mutex_wait_us":6133,"prepare.queue_time_us":2205,"prepare.run_cpu_time_us":9234,"prepare.run_wall_time_us":155654,"raft.queue_time_us":6410,"raft.run_cpu_time_us":887,"raft.run_wall_time_us":886,"replication_time_us":804157,"thread_start_us":170458,"threads_started":2,"wal-append.queue_time_us":128737}]]}
I20260923 17:31:32.342877 18241 leader_election.cc:288] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:32.585330 16381 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 354f66c852b94b1d9f44b6bc012cf81b consensus state has no leader
I20260923 17:31:32.587056 16381 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_81951dda1cdb4e91a19d5fec01501dff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:32.588914 16381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81951dda1cdb4e91a19d5fec01501dff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:32.594208 17146 raft_consensus.cc:348] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.594957 17146 raft_consensus.cc:374] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.595197 17146 raft_consensus.cc:729] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.598176 17146 consensus_queue.cc:260] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.600397 17146 ts_tablet_manager.cc:1428] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.060s user 0.003s sys 0.003s
I20260923 17:31:32.601131 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:32.615334 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217"
I20260923 17:31:32.624814 17143 tablet_bootstrap.cc:654] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.629949 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 5c929b8beaaf4c7aa668852b63af2204 (table default.Table_8be9935ea009465aaaae2c81f5fa6783 [id=aaedb0c6cff242f2acaf82970a6c0e30]) successfully deleted
I20260923 17:31:32.630353 17146 tablet_bootstrap.cc:492] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:32.634276 18244 consensus_queue.cc:1035] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.634876 18204 consensus_queue.cc:1035] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.632164 16678 leader_election.cc:302] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:32.636461 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet df53a9825e5348189485f30b8fdedc63 because the tablet was not found. No further retry: Not found: Tablet not found: df53a9825e5348189485f30b8fdedc63
I20260923 17:31:32.637169 16678 leader_election.cc:302] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:32.637440 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd because the tablet was not found. No further retry: Not found: Tablet not found: 19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:32.637948 18245 raft_consensus.cc:2781] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:32.638236 18245 raft_consensus.cc:482] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:32.638520 18245 raft_consensus.cc:3037] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:32.639000 18190 consensus_queue.cc:1035] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.640202 18190 raft_consensus.cc:2781] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:32.640525 18190 raft_consensus.cc:482] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:32.640810 18190 raft_consensus.cc:3037] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:32.607673 16384 catalog_manager.cc:5495] T c1d71b22338241839b184e50d7fdd8f6 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:32.643144 18211 consensus_queue.cc:1046] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.644238 18012 raft_consensus.cc:3032] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Stepping down as leader of term 1
I20260923 17:31:32.644557 18012 raft_consensus.cc:729] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:32.644371 16384 catalog_manager.cc:5495] T adc1f188f61e40a2ae42fcb716f5619c P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:32.645273 18012 consensus_queue.cc:260] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.648957 18255 raft_consensus.cc:482] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.649545 18255 raft_consensus.cc:504] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.652985 18255 leader_election.cc:288] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.654520 18012 raft_consensus.cc:3037] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Advancing to term 2
I20260923 17:31:32.594004 18221 ts_tablet_manager.cc:1939] T 5c929b8beaaf4c7aa668852b63af2204 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:32.503027 18167 raft_consensus.cc:482] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.657181 18167 raft_consensus.cc:504] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.667258 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b11a6fd84840178b81d2b7610c7351" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:32.674351 18245 raft_consensus.cc:504] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.677518 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b11a6fd84840178b81d2b7610c7351" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:32.688736 18245 leader_election.cc:288] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:32.690335 18167 leader_election.cc:288] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.700363 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 228) took 1245 ms. Trace:
I20260923 17:31:32.700868 18221 rpcz_store.cc:276] 0923 17:31:31.454573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.454710 (+ 137us) service_pool.cc:227] Handling call
0923 17:31:31.644338 (+189628us) ts_tablet_manager.cc:1145] Deleting tablet 5c929b8beaaf4c7aa668852b63af2204
0923 17:31:31.673916 (+ 29578us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:32.069751 (+395835us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:32.186430 (+116679us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:32.700283 (+513853us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":24704,"tablet-delete.queue_time_us":117010,"thread_start_us":116951,"threads_started":1}
I20260923 17:31:32.702850 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 5c929b8beaaf4c7aa668852b63af2204 (table default.Table_8be9935ea009465aaaae2c81f5fa6783 [id=aaedb0c6cff242f2acaf82970a6c0e30]) successfully deleted
I20260923 17:31:32.704043 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 233) took 1197 ms. Trace:
I20260923 17:31:32.704485 18221 rpcz_store.cc:276] 0923 17:31:31.506752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.506936 (+ 184us) service_pool.cc:227] Handling call
0923 17:31:32.703716 (+1196780us) ts_tablet_manager.cc:1145] Deleting tablet 19ebfb3cfd3d49b39c7cf453199ac8bd
0923 17:31:32.703765 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:32.703980 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1175884}
W20260923 17:31:32.705950 16826 leader_election.cc:341] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: 547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:32.706344 16826 leader_election.cc:302] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:32.708107 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd because the tablet was not found. No further retry: Not found: Tablet not found: 19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:32.709426 18167 raft_consensus.cc:2726] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.647393 16384 catalog_manager.cc:5495] T 254793351e5245af9a4fa1642cad5e82 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:32.713248 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:32.714138 16384 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:32.697816 18190 raft_consensus.cc:504] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.730489 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:32.732996 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:32.745996 18012 raft_consensus.cc:482] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.746605 18012 raft_consensus.cc:504] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.748992 18012 leader_election.cc:288] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:32.758683 16884 raft_consensus.cc:3037] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:32.765559 17904 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c36e79694e154f84a46980719217d7f2" instance_seqno: 1790184665320257) as {username='slave'} at 127.15.246.130:59589; Asking this server to re-register.
I20260923 17:31:32.768649 18222 raft_consensus.cc:2229] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:32.769562 18222 raft_consensus.cc:2258] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:32.775789 18190 leader_election.cc:288] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:32.778399 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 354f66c852b94b1d9f44b6bc012cf81b consensus state has no leader
I20260923 17:31:32.660301 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.660920 17146 tablet_bootstrap.cc:654] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.780701 17040 raft_consensus.cc:2379] T a71e9d28008c4670b3d48f32161d6aee P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c36e79694e154f84a46980719217d7f2 in current term 1: Already voted for candidate be93fda0843b44f5abc49e6f35c69217 in this term.
I20260923 17:31:32.781661 16893 raft_consensus.cc:3037] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:32.668785 16972 leader_election.cc:341] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:32.782866 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 114139us
W20260923 17:31:32.783619 16972 leader_election.cc:341] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.783972 16972 leader_election.cc:302] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:32.785457 18248 raft_consensus.cc:2726] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.785555 18221 tablet_replica.cc:330] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:32.786502 18221 raft_consensus.cc:2229] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:32.793145 17907 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" instance_seqno: 1790184663105309) as {username='slave'} at 127.15.246.129:47627; Asking this server to re-register.
I20260923 17:31:32.794536 18221 raft_consensus.cc:2258] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:32.794045 17143 tablet_bootstrap.cc:492] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:32.798652 17144 tablet_bootstrap.cc:492] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:32.799208 17144 ts_tablet_manager.cc:1397] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.403s user 0.020s sys 0.000s
I20260923 17:31:32.797067 17143 ts_tablet_manager.cc:1397] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.375s user 0.009s sys 0.003s
I20260923 17:31:32.772050 18222 ts_tablet_manager.cc:1905] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:32.802567 17143 raft_consensus.cc:348] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.803269 17143 raft_consensus.cc:374] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.806861 18248 raft_consensus.cc:482] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:32.807313 18248 raft_consensus.cc:504] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.809751 18248 leader_election.cc:288] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:32.822008 17143 raft_consensus.cc:729] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.810377 16888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feebaa76bcd4ff5b707814191cb3341" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:32.822724 17143 consensus_queue.cc:260] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:32.835335 17143 ts_tablet_manager.cc:1428] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.035s user 0.001s sys 0.005s
I20260923 17:31:32.836382 17143 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:32.836117 16972 leader_election.cc:341] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.821254 16938 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:32.821682 18221 ts_tablet_manager.cc:1905] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:32.850178 17146 tablet_bootstrap.cc:492] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:32.850580 17146 ts_tablet_manager.cc:1397] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.220s user 0.011s sys 0.008s
I20260923 17:31:32.851467 16678 leader_election.cc:302] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:32.852613 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feebaa76bcd4ff5b707814191cb3341" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:32.848148 18265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:32.856464 18265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:32.859549 17144 raft_consensus.cc:348] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.860343 17144 raft_consensus.cc:374] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.860574 17144 raft_consensus.cc:729] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.861145 17144 consensus_queue.cc:260] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.864686 17146 raft_consensus.cc:348] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.871274 17146 raft_consensus.cc:374] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:32.871845 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 270 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:32.872551 17146 raft_consensus.cc:729] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:32.873157 17144 ts_tablet_manager.cc:1428] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.074s user 0.006s sys 0.000s
I20260923 17:31:32.874207 16478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 270 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:32.874940 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71e9d28008c4670b3d48f32161d6aee" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:32.875785 17830 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:32.879902 18265 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:32.880539 17146 consensus_queue.cc:260] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:32.883493 18204 raft_consensus.cc:2781] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:32.883890 18204 raft_consensus.cc:482] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:32.884197 18204 raft_consensus.cc:3037] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:32.884531 18265 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260923 17:31:32.886988 17146 ts_tablet_manager.cc:1428] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
W20260923 17:31:32.874228 16972 leader_election.cc:341] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:32.888284 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:32.893806 16972 leader_election.cc:302] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:32.893941 17039 raft_consensus.cc:3037] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:32.894469 18255 raft_consensus.cc:2726] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:32.901216 17039 raft_consensus.cc:2454] T cfa712f9362e416e9cc6defcd8198bcd P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:32.904534 18190 consensus_queue.cc:1035] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:32.910275 17027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:32.913247 17146 tablet_bootstrap.cc:492] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:32.914450 18211 consensus_queue.cc:1035] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:32.068122 17936 rpcz_store.cc:276] 0923 17:31:28.581550 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.581660 (+ 110us) service_pool.cc:227] Handling call
0923 17:31:29.084545 (+502885us) raft_consensus.cc:1457] Updating replica for 266 ops
0923 17:31:29.144749 (+ 60204us) raft_consensus.cc:1520] Early marking committed up to index 0
0923 17:31:29.144794 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 266 ops
0923 17:31:29.145522 (+ 728us) write_op.cc:270] Start()
0923 17:31:29.145660 (+ 138us) write_op.cc:276] Timestamp: P: 1790184667521489 usec, L: 0
0923 17:31:29.188187 (+ 42527us) write_op.cc:270] Start()
0923 17:31:29.188305 (+ 118us) write_op.cc:276] Timestamp: P: 1790184667536746 usec, L: 0
0923 17:31:29.191996 (+ 3691us) write_op.cc:270] Start()
0923 17:31:29.192131 (+ 135us) write_op.cc:276] Timestamp: P: 1790184667542368 usec, L: 0
0923 17:31:29.192623 (+ 492us) write_op.cc:270] Start()
0923 17:31:29.192696 (+ 73us) write_op.cc:276] Timestamp: P: 1790184667549087 usec, L: 0
0923 17:31:29.193144 (+ 448us) write_op.cc:270] Start()
0923 17:31:29.193218 (+ 74us) write_op.cc:276] Timestamp: P: 1790184667578326 usec, L: 0
0923 17:31:29.193609 (+ 391us) write_op.cc:270] Start()
0923 17:31:29.193693 (+ 84us) write_op.cc:276] Timestamp: P: 1790184667603334 usec, L: 0
0923 17:31:29.194109 (+ 416us) write_op.cc:270] Start()
0923 17:31:29.194236 (+ 127us) write_op.cc:276] Timestamp: P: 1790184667605324 usec, L: 0
0923 17:31:29.194634 (+ 398us) write_op.cc:270] Start()
0923 17:31:29.194711 (+ 77us) write_op.cc:276] Timestamp: P: 1790184667607099 usec, L: 0
0923 17:31:29.209291 (+ 14580us) write_op.cc:270] Start()
0923 17:31:29.209400 (+ 109us) write_op.cc:276] Timestamp: P: 1790184667611822 usec, L: 0
0923 17:31:29.211250 (+ 1850us) write_op.cc:270] Start()
0923 17:31:29.211331 (+ 81us) write_op.cc:276] Timestamp: P: 1790184667668191 usec, L: 0
0923 17:31:29.214786 (+ 3455us) write_op.cc:270] Start()
0923 17:31:29.214899 (+ 113us) write_op.cc:276] Timestamp: P: 1790184667783257 usec, L: 0
0923 17:31:29.216524 (+ 1625us) write_op.cc:270] Start()
0923 17:31:29.216623 (+ 99us) write_op.cc:276] Timestamp: P: 1790184667857698 usec, L: 0
0923 17:31:29.217107 (+ 484us) write_op.cc:270] Start()
0923 17:31:29.217178 (+ 71us) write_op.cc:276] Timestamp: P: 1790184667894602 usec, L: 0
0923 17:31:29.217588 (+ 410us) write_op.cc:270] Start()
0923 17:31:29.217678 (+ 90us) write_op.cc:276] Timestamp: P: 1790184667947448 usec, L: 0
0923 17:31:29.218085 (+ 407us) write_op.cc:270] Start()
0923 17:31:29.218155 (+ 70us) write_op.cc:276] Timestamp: P: 1790184667999654 usec, L: 0
0923 17:31:29.267123 (+ 48968us) write_op.cc:270] Start()
0923 17:31:29.267613 (+ 490us) write_op.cc:276] Timestamp: P: 1790184668026436 usec, L: 0
0923 17:31:29.272368 (+ 4755us) write_op.cc:270] Start()
0923 17:31:29.273601 (+ 1233us) write_op.cc:276] Timestamp: P: 1790184668037454 usec, L: 0
0923 17:31:29.316195 (+ 42594us) write_op.cc:270] Start()
0923 17:31:29.316428 (+ 233us) write_op.cc:276] Timestamp: P: 1790184668098165 usec, L: 0
0923 17:31:29.331560 (+ 15132us) write_op.cc:270] Start()
0923 17:31:29.331677 (+ 117us) write_op.cc:276] Timestamp: P: 1790184668139314 usec, L: 0
0923 17:31:29.341110 (+ 9433us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b5400010280. stack: 00007fb69a1c6033 00007fb69a1c5f30 00007fb699d47534 00000000004db2b4 00000000004db249 00007fb6a14e2361 00007fb6a15b2a86 00007fb6a15a8e03 00007fb6a156ff96 00007fb6a156facf 00007fb6a1144040 00007fb6a114d536 00007fb6a114c99e 00007fb6a21292e9 00007fb6a08ff684 00007fb6a08ff612
0923 17:31:29.341920 (+ 810us) write_op.cc:270] Start()
0923 17:31:29.342041 (+ 121us) write_op.cc:276] Timestamp: P: 1790184668157533 usec, L: 0
0923 17:31:29.342776 (+ 735us) write_op.cc:270] Start()
0923 17:31:29.342951 (+ 175us) write_op.cc:276] Timestamp: P: 1790184668203901 usec, L: 0
0923 17:31:29.343767 (+ 816us) write_op.cc:270] Start()
0923 17:31:29.343849 (+ 82us) write_op.cc:276] Timestamp: P: 1790184668237406 usec, L: 0
0923 17:31:29.356715 (+ 12866us) write_op.cc:270] Start()
0923 17:31:29.357430 (+ 715us) write_op.cc:276] Timestamp: P: 1790184668260745 usec, L: 0
0923 17:31:29.361619 (+ 4189us) write_op.cc:270] Start()
0923 17:31:29.362451 (+ 832us) write_op.cc:276] Timestamp: P: 1790184668370293 usec, L: 0
0923 17:31:29.410488 (+ 48037us) write_op.cc:270] Start()
0923 17:31:29.410647 (+ 159us) write_op.cc:276] Timestamp: P: 1790184668418849 usec, L: 0
0923 17:31:29.411737 (+ 1090us) write_op.cc:270] Start()
0923 17:31:29.411850 (+ 113us) write_op.cc:276] Timestamp: P: 1790184668458425 usec, L: 0
0923 17:31:29.412562 (+ 712us) write_op.cc:270] Start()
0923 17:31:29.412718 (+ 156us) write_op.cc:276] Timestamp: P: 1790184668493390 usec, L: 0
0923 17:31:29.413166 (+ 448us) write_op.cc:270] Start()
0923 17:31:29.413241 (+ 75us) write_op.cc:276] Timestamp: P: 1790184668502327 usec, L: 0
0923 17:31:29.428944 (+ 15703us) write_op.cc:270] Start()
0923 17:31:29.429122 (+ 178us) write_op.cc:276] Timestamp: P: 1790184668526669 usec, L: 0
0923 17:31:29.430201 (+ 1079us) write_op.cc:270] Start()
0923 17:31:29.430393 (+ 192us) write_op.cc:276] Timestamp: P: 1790184668536998 usec, L: 0
0923 17:31:29.431516 (+ 1123us) write_op.cc:270] Start()
0923 17:31:29.431674 (+ 158us) write_op.cc:276] Timestamp: P: 1790184668545040 usec, L: 0
0923 17:31:29.446621 (+ 14947us) write_op.cc:270] Start()
0923 17:31:29.446735 (+ 114us) write_op.cc:276] Timestamp: P: 1790184668685673 usec, L: 0
0923 17:31:29.447352 (+ 617us) write_op.cc:270] Start()
0923 17:31:29.447426 (+ 74us) write_op.cc:276] Timestamp: P: 1790184668784270 usec, L: 0
0923 17:31:29.447863 (+ 437us) write_op.cc:270] Start()
0923 17:31:29.447941 (+ 78us) write_op.cc:276] Timestamp: P: 1790184668857188 usec, L: 0
0923 17:31:29.448395 (+ 454us) write_op.cc:270] Start()
0923 17:31:29.448481 (+ 86us) write_op.cc:276] Timestamp: P: 1790184668922300 usec, L: 0
0923 17:31:29.448911 (+ 430us) write_op.cc:270] Start()
0923 17:31:29.448986 (+ 75us) write_op.cc:276] Timestamp: P: 1790184668963334 usec, L: 0
0923 17:31:29.449410 (+ 424us) write_op.cc:270] Start()
0923 17:31:29.449483 (+ 73us) write_op.cc:276] Timestamp: P: 1790184668970290 usec, L: 0
0923 17:31:29.449952 (+ 469us) write_op.cc:270] Start()
0923 17:31:29.450034 (+ 82us) write_op.cc:276] Timestamp: P: 1790184668972980 usec, L: 0
0923 17:31:29.450685 (+ 651us) write_op.cc:270] Start()
0923 17:31:29.450763 (+ 78us) write_op.cc:276] Timestamp: P: 1790184668974924 usec, L: 0
0923 17:31:29.451183 (+ 420us) write_op.cc:270] Start()
0923 17:31:29.451254 (+ 71us) write_op.cc:276] Timestamp: P: 1790184669040356 usec, L: 0
0923 17:31:29.451874 (+ 620us) write_op.cc:270] Start()
0923 17:31:29.451999 (+ 125us) write_op.cc:276] Timestamp: P: 1790184669082026 usec, L: 0
0923 17:31:29.504609 (+ 52610us) write_op.cc:270] Start()
0923 17:31:29.504703 (+ 94us) write_op.cc:276] Timestamp: P: 1790184669115603 usec, L: 0
0923 17:31:29.515458 (+ 10755us) write_op.cc:270] Start()
0923 17:31:29.515566 (+ 108us) write_op.cc:276] Timestamp: P: 1790184669133831 usec, L: 0
0923 17:31:29.516071 (+ 505us) write_op.cc:270] Start()
0923 17:31:29.538261 (+ 22190us) write_op.cc:276] Timestamp: P: 1790184669199596 usec, L: 0
0923 17:31:29.538830 (+ 569us) write_op.cc:270] Start()
0923 17:31:29.538916 (+ 86us) write_op.cc:276] Timestamp: P: 1790184669201910 usec, L: 0
0923 17:31:29.539358 (+ 442us) write_op.cc:270] Start()
0923 17:31:29.539465 (+ 107us) write_op.cc:276] Timestamp: P: 1790184669329551 usec, L: 0
0923 17:31:29.539878 (+ 413us) write_op.cc:270] Start()
0923 17:31:29.539955 (+ 77us) write_op.cc:276] Timestamp: P: 1790184669449350 usec, L: 0
0923 17:31:29.540373 (+ 418us) write_op.cc:270] Start()
0923 17:31:29.540544 (+ 171us) write_op.cc:276] Timestamp: P: 1790184669467900 usec, L: 0
0923 17:31:29.541024 (+ 480us) write_op.cc:270] Start()
0923 17:31:29.541099 (+ 75us) write_op.cc:276] Timestamp: P: 1790184669516100 usec, L: 0
0923 17:31:29.541496 (+ 397us) write_op.cc:270] Start()
0923 17:31:29.541568 (+ 72us) write_op.cc:276] Timestamp: P: 1790184669517771 usec, L: 0
0923 17:31:29.541959 (+ 391us) write_op.cc:270] Start()
0923 17:31:29.542029 (+ 70us) write_op.cc:276] Timestamp: P: 1790184669522779 usec, L: 0
0923 17:31:29.542413 (+ 384us) write_op.cc:270] Start()
0923 17:31:29.542493 (+ 80us) write_op.cc:276] Timestamp: P: 1790184669606972 usec, L: 0
0923 17:31:29.542910 (+ 417us) write_op.cc:270] Start()
0923 17:31:29.542979 (+ 69us) write_op.cc:276] Timestamp: P: 1790184669633905 usec, L: 0
0923 17:31:29.586272 (+ 43293us) write_op.cc:270] Start()
0923 17:31:29.586390 (+ 118us) write_op.cc:276] Timestamp: P: 1790184669668073 usec, L: 0
0923 17:31:29.587529 (+ 1139us) write_op.cc:270] Start()
0923 17:31:29.587651 (+ 122us) write_op.cc:276] Timestamp: P: 1790184669683674 usec, L: 0
0923 17:31:29.588262 (+ 611us) write_op.cc:270] Start()
0923 17:31:29.588375 (+ 113us) write_op.cc:276] Timestamp: P: 1790184669685702 usec, L: 0
0923 17:31:29.589049 (+ 674us) write_op.cc:270] Start()
0923 17:31:29.589211 (+ 162us) write_op.cc:276] Timestamp: P: 1790184669700128 usec, L: 0
0923 17:31:29.602902 (+ 13691us) write_op.cc:270] Start()
0923 17:31:29.603032 (+ 130us) write_op.cc:276] Timestamp: P: 1790184669725240 usec, L: 0
0923 17:31:29.610583 (+ 7551us) write_op.cc:270] Start()
0923 17:31:29.614500 (+ 3917us) write_op.cc:276] Timestamp: P: 1790184669769737 usec, L: 0
0923 17:31:29.620801 (+ 6301us) write_op.cc:270] Start()
0923 17:31:29.621075 (+ 274us) write_op.cc:276] Timestamp: P: 1790184669779506 usec, L: 0
0923 17:31:29.636610 (+ 15535us) write_op.cc:270] Start()
0923 17:31:29.636708 (+ 98us) write_op.cc:276] Timestamp: P: 1790184669894042 usec, L: 0
0923 17:31:29.638683 (+ 1975us) write_op.cc:270] Start()
0923 17:31:29.638766 (+ 83us) write_op.cc:276] Timestamp: P: 1790184669939806 usec, L: 0
0923 17:31:29.639187 (+ 421us) write_op.cc:270] Start()
0923 17:31:29.639260 (+ 73us) write_op.cc:276] Timestamp: P: 1790184669953349 usec, L: 0
0923 17:31:29.639678 (+ 418us) write_op.cc:270] Start()
0923 17:31:29.639750 (+ 72us) write_op.cc:276] Timestamp: P: 1790184669992072 usec, L: 0
0923 17:31:29.640357 (+ 607us) write_op.cc:270] Start()
0923 17:31:29.640441 (+ 84us) write_op.cc:276] Timestamp: P: 1790184670141386 usec, L: 0
0923 17:31:29.640950 (+ 509us) write_op.cc:270] Start()
0923 17:31:29.641087 (+ 137us) write_op.cc:276] Timestamp: P: 1790184670143586 usec, L: 0
0923 17:31:29.642229 (+ 1142us) write_op.cc:270] Start()
0923 17:31:29.642403 (+ 174us) write_op.cc:276] Timestamp: P: 1790184670362288 usec, L: 0
0923 17:31:29.642869 (+ 466us) write_op.cc:270] Start()
0923 17:31:29.643003 (+ 134us) write_op.cc:276] Timestamp: P: 1790184670368850 usec, L: 0
0923 17:31:29.721594 (+ 78591us) write_op.cc:270] Start()
0923 17:31:29.721697 (+ 103us) write_op.cc:276] Timestamp: P: 1790184670371042 usec, L: 0
0923 17:31:29.722243 (+ 546us) write_op.cc:270] Start()
0923 17:31:29.722476 (+ 233us) write_op.cc:276] Timestamp: P: 1790184670376444 usec, L: 0
0923 17:31:29.730615 (+ 8139us) write_op.cc:270] Start()
0923 17:31:29.730802 (+ 187us) write_op.cc:276] Timestamp: P: 1790184670378608 usec, L: 0
0923 17:31:29.733153 (+ 2351us) write_op.cc:270] Start()
0923 17:31:29.733242 (+ 89us) write_op.cc:276] Timestamp: P: 1790184670396736 usec, L: 0
0923 17:31:29.733689 (+ 447us) write_op.cc:270] Start()
0923 17:31:29.733763 (+ 74us) write_op.cc:276] Timestamp: P: 1790184670514256 usec, L: 0
0923 17:31:29.734188 (+ 425us) write_op.cc:270] Start()
0923 17:31:29.734268 (+ 80us) write_op.cc:276] Timestamp: P: 1790184670610411 usec, L: 0
0923 17:31:29.744325 (+ 10057us) write_op.cc:270] Start()
0923 17:31:29.744422 (+ 97us) write_op.cc:276] Timestamp: P: 1790184670657974 usec, L: 0
0923 17:31:29.744933 (+ 511us) write_op.cc:270] Start()
0923 17:31:29.745014 (+ 81us) write_op.cc:276] Timestamp: P: 1790184670803011 usec, L: 0
0923 17:31:29.745433 (+ 419us) write_op.cc:270] Start()
0923 17:31:29.745510 (+ 77us) write_op.cc:276] Timestamp: P: 1790184670804799 usec, L: 0
0923 17:31:29.745951 (+ 441us) write_op.cc:270] Start()
0923 17:31:29.746029 (+ 78us) write_op.cc:276] Timestamp: P: 1790184670806818 usec, L: 0
0923 17:31:29.746448 (+ 419us) write_op.cc:270] Start()
0923 17:31:29.746521 (+ 73us) write_op.cc:276] Timestamp: P: 1790184670833833 usec, L: 0
0923 17:31:29.746944 (+ 423us) write_op.cc:270] Start()
0923 17:31:29.747014 (+ 70us) write_op.cc:276] Timestamp: P: 1790184670842509 usec, L: 0
0923 17:31:29.747416 (+ 402us) write_op.cc:270] Start()
0923 17:31:29.747504 (+ 88us) write_op.cc:276] Timestamp: P: 1790184670938865 usec, L: 0
0923 17:31:29.747917 (+ 413us) write_op.cc:270] Start()
0923 17:31:29.747994 (+ 77us) write_op.cc:276] Timestamp: P: 1790184670982622 usec, L: 0
0923 17:31:29.763522 (+ 15528us) write_op.cc:270] Start()
0923 17:31:29.763618 (+ 96us) write_op.cc:276] Timestamp: P: 1790184671014208 usec, L: 0
0923 17:31:29.764156 (+ 538us) write_op.cc:270] Start()
0923 17:31:29.764253 (+ 97us) write_op.cc:276] Timestamp: P: 1790184671131525 usec, L: 0
0923 17:31:29.764704 (+ 451us) write_op.cc:270] Start()
0923 17:31:29.764777 (+ 73us) write_op.cc:276] Timestamp: P: 1790184671134006 usec, L: 0
0923 17:31:29.765196 (+ 419us) write_op.cc:270] Start()
0923 17:31:29.765269 (+ 73us) write_op.cc:276] Timestamp: P: 1790184671155307 usec, L: 0
0923 17:31:29.765670 (+ 401us) write_op.cc:270] Start()
0923 17:31:29.765746 (+ 76us) write_op.cc:276] Timestamp: P: 1790184671315453 usec, L: 0
0923 17:31:29.766167 (+ 421us) write_op.cc:270] Start()
0923 17:31:29.766239 (+ 72us) write_op.cc:276] Timestamp: P: 1790184671337704 usec, L: 0
0923 17:31:29.766631 (+ 392us) write_op.cc:270] Start()
0923 17:31:29.766728 (+ 97us) write_op.cc:276] Timestamp: P: 1790184671387932 usec, L: 0
0923 17:31:29.777743 (+ 11015us) write_op.cc:270] Start()
0923 17:31:29.777844 (+ 101us) write_op.cc:276] Timestamp: P: 1790184671429801 usec, L: 0
0923 17:31:29.778322 (+ 478us) write_op.cc:270] Start()
0923 17:31:29.778400 (+ 78us) write_op.cc:276] Timestamp: P: 1790184671480685 usec, L: 0
0923 17:31:29.778829 (+ 429us) write_op.cc:270] Start()
0923 17:31:29.778915 (+ 86us) write_op.cc:276] Timestamp: P: 1790184671624668 usec, L: 0
0923 17:31:29.779402 (+ 487us) write_op.cc:270] Start()
0923 17:31:29.779494 (+ 92us) write_op.cc:276] Timestamp: P: 1790184671626782 usec, L: 0
0923 17:31:29.779938 (+ 444us) write_op.cc:270] Start()
0923 17:31:29.780010 (+ 72us) write_op.cc:276] Timestamp: P: 1790184671756993 usec, L: 0
0923 17:31:29.812093 (+ 32083us) write_op.cc:270] Start()
0923 17:31:29.812209 (+ 116us) write_op.cc:276] Timestamp: P: 1790184671893911 usec, L: 0
0923 17:31:29.817029 (+ 4820us) write_op.cc:270] Start()
0923 17:31:29.817115 (+ 86us) write_op.cc:276] Timestamp: P: 1790184671927496 usec, L: 0
0923 17:31:29.817539 (+ 424us) write_op.cc:270] Start()
0923 17:31:29.817611 (+ 72us) write_op.cc:276] Timestamp: P: 1790184671990908 usec, L: 0
0923 17:31:29.818024 (+ 413us) write_op.cc:270] Start()
0923 17:31:29.818207 (+ 183us) write_op.cc:276] Timestamp: P: 1790184672025717 usec, L: 0
0923 17:31:29.818691 (+ 484us) write_op.cc:270] Start()
0923 17:31:29.818768 (+ 77us) write_op.cc:276] Timestamp: P: 1790184672045064 usec, L: 0
0923 17:31:29.819203 (+ 435us) write_op.cc:270] Start()
0923 17:31:29.819276 (+ 73us) write_op.cc:276] Timestamp: P: 1790184672094706 usec, L: 0
0923 17:31:29.819678 (+ 402us) write_op.cc:270] Start()
0923 17:31:29.819750 (+ 72us) write_op.cc:276] Timestamp: P: 1790184672157750 usec, L: 0
0923 17:31:29.840790 (+ 21040us) write_op.cc:270] Start()
0923 17:31:29.840883 (+ 93us) write_op.cc:276] Timestamp: P: 1790184672164046 usec, L: 0
0923 17:31:29.841419 (+ 536us) write_op.cc:270] Start()
0923 17:31:29.841500 (+ 81us) write_op.cc:276] Timestamp: P: 1790184672216866 usec, L: 0
0923 17:31:29.853738 (+ 12238us) write_op.cc:270] Start()
0923 17:31:29.853831 (+ 93us) write_op.cc:276] Timestamp: P: 1790184672296774 usec, L: 0
0923 17:31:29.854319 (+ 488us) write_op.cc:270] Start()
0923 17:31:29.854386 (+ 67us) write_op.cc:276] Timestamp: P: 1790184672299011 usec, L: 0
0923 17:31:29.854822 (+ 436us) write_op.cc:270] Start()
0923 17:31:29.854890 (+ 68us) write_op.cc:276] Timestamp: P: 1790184672427456 usec, L: 0
0923 17:31:29.855289 (+ 399us) write_op.cc:270] Start()
0923 17:31:29.855366 (+ 77us) write_op.cc:276] Timestamp: P: 1790184672470624 usec, L: 0
0923 17:31:29.855799 (+ 433us) write_op.cc:270] Start()
0923 17:31:29.855881 (+ 82us) write_op.cc:276] Timestamp: P: 1790184672477468 usec, L: 0
0923 17:31:29.856339 (+ 458us) write_op.cc:270] Start()
0923 17:31:29.856416 (+ 77us) write_op.cc:276] Timestamp: P: 1790184672488136 usec, L: 0
0923 17:31:29.856847 (+ 431us) write_op.cc:270] Start()
0923 17:31:29.856922 (+ 75us) write_op.cc:276] Timestamp: P: 1790184672491165 usec, L: 0
0923 17:31:29.869068 (+ 12146us) write_op.cc:270] Start()
0923 17:31:29.869154 (+ 86us) write_op.cc:276] Timestamp: P: 1790184672522067 usec, L: 0
0923 17:31:29.869582 (+ 428us) write_op.cc:270] Start()
0923 17:31:29.869648 (+ 66us) write_op.cc:276] Timestamp: P: 1790184672526000 usec, L: 0
0923 17:31:29.870040 (+ 392us) write_op.cc:270] Start()
0923 17:31:29.870108 (+ 68us) write_op.cc:276] Timestamp: P: 1790184672576020 usec, L: 0
0923 17:31:29.870505 (+ 397us) write_op.cc:270] Start()
0923 17:31:29.870582 (+ 77us) write_op.cc:276] Timestamp: P: 1790184672654487 usec, L: 0
0923 17:31:29.871070 (+ 488us) write_op.cc:270] Start()
0923 17:31:29.871144 (+ 74us) write_op.cc:276] Timestamp: P: 1790184672747294 usec, L: 0
0923 17:31:29.871641 (+ 497us) write_op.cc:270] Start()
0923 17:31:29.871791 (+ 150us) write_op.cc:276] Timestamp: P: 1790184672772472 usec, L: 0
0923 17:31:29.896672 (+ 24881us) write_op.cc:270] Start()
0923 17:31:29.896774 (+ 102us) write_op.cc:276] Timestamp: P: 1790184672808257 usec, L: 0
0923 17:31:29.897605 (+ 831us) write_op.cc:270] Start()
0923 17:31:29.897704 (+ 99us) write_op.cc:276] Timestamp: P: 1790184672899921 usec, L: 0
0923 17:31:29.898304 (+ 600us) write_op.cc:270] Start()
0923 17:31:29.898381 (+ 77us) write_op.cc:276] Timestamp: P: 1790184672967624 usec, L: 0
0923 17:31:29.898867 (+ 486us) write_op.cc:270] Start()
0923 17:31:29.898994 (+ 127us) write_op.cc:276] Timestamp: P: 1790184673042972 usec, L: 0
0923 17:31:29.899475 (+ 481us) write_op.cc:270] Start()
0923 17:31:29.899585 (+ 110us) write_op.cc:276] Timestamp: P: 1790184673079192 usec, L: 0
0923 17:31:29.923382 (+ 23797us) write_op.cc:270] Start()
0923 17:31:29.923495 (+ 113us) write_op.cc:276] Timestamp: P: 1790184673164543 usec, L: 0
0923 17:31:29.924088 (+ 593us) write_op.cc:270] Start()
0923 17:31:29.924168 (+ 80us) write_op.cc:276] Timestamp: P: 1790184673324158 usec, L: 0
0923 17:31:29.924870 (+ 702us) write_op.cc:270] Start()
0923 17:31:29.924950 (+ 80us) write_op.cc:276] Timestamp: P: 1790184673327178 usec, L: 0
0923 17:31:29.927032 (+ 2082us) write_op.cc:270] Start()
0923 17:31:29.927187 (+ 155us) write_op.cc:276] Timestamp: P: 1790184673427861 usec, L: 0
0923 17:31:29.928021 (+ 834us) write_op.cc:270] Start()
0923 17:31:29.928165 (+ 144us) write_op.cc:276] Timestamp: P: 1790184673494638 usec, L: 0
0923 17:31:29.941090 (+ 12925us) write_op.cc:270] Start()
0923 17:31:29.941261 (+ 171us) write_op.cc:276] Timestamp: P: 1790184673595969 usec, L: 0
0923 17:31:29.957915 (+ 16654us) write_op.cc:270] Start()
0923 17:31:29.958170 (+ 255us) write_op.cc:276] Timestamp: P: 1790184673657638 usec, L: 0
0923 17:31:29.959010 (+ 840us) write_op.cc:270] Start()
0923 17:31:29.959178 (+ 168us) write_op.cc:276] Timestamp: P: 1790184673716638 usec, L: 0
0923 17:31:29.959741 (+ 563us) write_op.cc:270] Start()
0923 17:31:29.959818 (+ 77us) write_op.cc:276] Timestamp: P: 1790184673779049 usec, L: 0
0923 17:31:29.960311 (+ 493us) write_op.cc:270] Start()
0923 17:31:29.960389 (+ 78us) write_op.cc:276] Timestamp: P: 1790184673866052 usec, L: 0
0923 17:31:29.961449 (+ 1060us) write_op.cc:270] Start()
0923 17:31:29.961585 (+ 136us) write_op.cc:276] Timestamp: P: 1790184673888145 usec, L: 0
0923 17:31:29.962408 (+ 823us) write_op.cc:270] Start()
0923 17:31:29.962555 (+ 147us) write_op.cc:276] Timestamp: P: 1790184674000234 usec, L: 0
0923 17:31:29.963476 (+ 921us) write_op.cc:270] Start()
0923 17:31:29.963640 (+ 164us) write_op.cc:276] Timestamp: P: 1790184674151096 usec, L: 0
0923 17:31:29.976923 (+ 13283us) write_op.cc:270] Start()
0923 17:31:29.977039 (+ 116us) write_op.cc:276] Timestamp: P: 1790184674169612 usec, L: 0
0923 17:31:30.014161 (+ 37122us) write_op.cc:270] Start()
0923 17:31:30.014293 (+ 132us) write_op.cc:276] Timestamp: P: 1790184674222913 usec, L: 0
0923 17:31:30.016432 (+ 2139us) write_op.cc:270] Start()
0923 17:31:30.016534 (+ 102us) write_op.cc:276] Timestamp: P: 1790184674291165 usec, L: 0
0923 17:31:30.017065 (+ 531us) write_op.cc:270] Start()
0923 17:31:30.017145 (+ 80us) write_op.cc:276] Timestamp: P: 1790184674351010 usec, L: 0
0923 17:31:30.017723 (+ 578us) write_op.cc:270] Start()
0923 17:31:30.017855 (+ 132us) write_op.cc:276] Timestamp: P: 1790184674352894 usec, L: 0
0923 17:31:30.018398 (+ 543us) write_op.cc:270] Start()
0923 17:31:30.018480 (+ 82us) write_op.cc:276] Timestamp: P: 1790184674391156 usec, L: 0
0923 17:31:30.019083 (+ 603us) write_op.cc:270] Start()
0923 17:31:30.019293 (+ 210us) write_op.cc:276] Timestamp: P: 1790184674437677 usec, L: 0
0923 17:31:30.036757 (+ 17464us) write_op.cc:270] Start()
0923 17:31:30.036882 (+ 125us) write_op.cc:276] Timestamp: P: 1790184674494330 usec, L: 0
0923 17:31:30.039574 (+ 2692us) write_op.cc:270] Start()
0923 17:31:30.039673 (+ 99us) write_op.cc:276] Timestamp: P: 1790184674498349 usec, L: 0
0923 17:31:30.040246 (+ 573us) write_op.cc:270] Start()
0923 17:31:30.040320 (+ 74us) write_op.cc:276] Timestamp: P: 1790184674515728 usec, L: 0
0923 17:31:30.040748 (+ 428us) write_op.cc:270] Start()
0923 17:31:30.040818 (+ 70us) write_op.cc:276] Timestamp: P: 1790184674659287 usec, L: 0
0923 17:31:30.041221 (+ 403us) write_op.cc:270] Start()
0923 17:31:30.041287 (+ 66us) write_op.cc:276] Timestamp: P: 1790184675017779 usec, L: 0
0923 17:31:30.041644 (+ 357us) write_op.cc:270] Start()
0923 17:31:30.136663 (+ 95019us) write_op.cc:276] Timestamp: P: 1790184675047386 usec, L: 0
0923 17:31:30.137484 (+ 821us) write_op.cc:270] Start()
0923 17:31:30.137620 (+ 136us) write_op.cc:276] Timestamp: P: 1790184675127352 usec, L: 0
0923 17:31:30.138202 (+ 582us) write_op.cc:270] Start()
0923 17:31:30.138273 (+ 71us) write_op.cc:276] Timestamp: P: 1790184675163631 usec, L: 0
0923 17:31:30.138808 (+ 535us) write_op.cc:270] Start()
0923 17:31:30.138881 (+ 73us) write_op.cc:276] Timestamp: P: 1790184675221614 usec, L: 0
0923 17:31:30.139447 (+ 566us) write_op.cc:270] Start()
0923 17:31:30.139515 (+ 68us) write_op.cc:276] Timestamp: P: 1790184675327530 usec, L: 0
0923 17:31:30.140215 (+ 700us) write_op.cc:270] Start()
0923 17:31:30.233239 (+ 93024us) write_op.cc:276] Timestamp: P: 1790184675413352 usec, L: 0
0923 17:31:30.233816 (+ 577us) write_op.cc:270] Start()
0923 17:31:30.233918 (+ 102us) write_op.cc:276] Timestamp: P: 1790184675497096 usec, L: 0
0923 17:31:30.234345 (+ 427us) write_op.cc:270] Start()
0923 17:31:30.234410 (+ 65us) write_op.cc:276] Timestamp: P: 1790184675527119 usec, L: 0
0923 17:31:30.234832 (+ 422us) write_op.cc:270] Start()
0923 17:31:30.234900 (+ 68us) write_op.cc:276] Timestamp: P: 1790184675595716 usec, L: 0
0923 17:31:30.235396 (+ 496us) write_op.cc:270] Start()
0923 17:31:30.235461 (+ 65us) write_op.cc:276] Timestamp: P: 1790184675654890 usec, L: 0
0923 17:31:30.235865 (+ 404us) write_op.cc:270] Start()
0923 17:31:30.235927 (+ 62us) write_op.cc:276] Timestamp: P: 1790184675895971 usec, L: 0
0923 17:31:30.274087 (+ 38160us) write_op.cc:270] Start()
0923 17:31:30.274201 (+ 114us) write_op.cc:276] Timestamp: P: 1790184675979019 usec, L: 0
0923 17:31:30.274815 (+ 614us) write_op.cc:270] Start()
0923 17:31:30.274901 (+ 86us) write_op.cc:276] Timestamp: P: 1790184676136265 usec, L: 0
0923 17:31:30.312355 (+ 37454us) write_op.cc:270] Start()
0923 17:31:30.312474 (+ 119us) write_op.cc:276] Timestamp: P: 1790184676247393 usec, L: 0
0923 17:31:30.313089 (+ 615us) write_op.cc:270] Start()
0923 17:31:30.313165 (+ 76us) write_op.cc:276] Timestamp: P: 1790184676250598 usec, L: 0
0923 17:31:30.313612 (+ 447us) write_op.cc:270] Start()
0923 17:31:30.313681 (+ 69us) write_op.cc:276] Timestamp: P: 1790184676327963 usec, L: 0
0923 17:31:30.314228 (+ 547us) write_op.cc:270] Start()
0923 17:31:30.314310 (+ 82us) write_op.cc:276] Timestamp: P: 1790184676518681 usec, L: 0
0923 17:31:30.314923 (+ 613us) write_op.cc:270] Start()
0923 17:31:30.315008 (+ 85us) write_op.cc:276] Timestamp: P: 1790184676535264 usec, L: 0
0923 17:31:30.315402 (+ 394us) write_op.cc:270] Start()
0923 17:31:30.315473 (+ 71us) write_op.cc:276] Timestamp: P: 1790184676537078 usec, L: 0
0923 17:31:30.368257 (+ 52784us) write_op.cc:270] Start()
0923 17:31:30.368369 (+ 112us) write_op.cc:276] Timestamp: P: 1790184676538766 usec, L: 0
0923 17:31:30.369003 (+ 634us) write_op.cc:270] Start()
0923 17:31:30.369078 (+ 75us) write_op.cc:276] Timestamp: P: 1790184676624070 usec, L: 0
0923 17:31:30.369546 (+ 468us) write_op.cc:270] Start()
0923 17:31:30.369627 (+ 81us) write_op.cc:276] Timestamp: P: 1790184676774863 usec, L: 0
0923 17:31:30.370046 (+ 419us) write_op.cc:270] Start()
0923 17:31:30.370118 (+ 72us) write_op.cc:276] Timestamp: P: 1790184676792191 usec, L: 0
0923 17:31:30.370528 (+ 410us) write_op.cc:270] Start()
0923 17:31:30.370608 (+ 80us) write_op.cc:276] Timestamp: P: 1790184676887006 usec, L: 0
0923 17:31:30.370974 (+ 366us) write_op.cc:270] Start()
0923 17:31:30.371036 (+ 62us) write_op.cc:276] Timestamp: P: 1790184677008861 usec, L: 0
0923 17:31:30.371428 (+ 392us) write_op.cc:270] Start()
0923 17:31:30.371490 (+ 62us) write_op.cc:276] Timestamp: P: 1790184677058674 usec, L: 0
0923 17:31:30.371886 (+ 396us) write_op.cc:270] Start()
0923 17:31:30.371952 (+ 66us) write_op.cc:276] Timestamp: P: 1790184677107500 usec, L: 0
0923 17:31:30.562711 (+190759us) write_op.cc:270] Start()
0923 17:31:30.562820 (+ 109us) write_op.cc:276] Timestamp: P: 1790184677144693 usec, L: 0
0923 17:31:30.563474 (+ 654us) write_op.cc:270] Start()
0923 17:31:30.563552 (+ 78us) write_op.cc:276] Timestamp: P: 1790184677204702 usec, L: 0
0923 17:31:30.644406 (+ 80854us) write_op.cc:270] Start()
0923 17:31:30.644521 (+ 115us) write_op.cc:276] Timestamp: P: 1790184677643122 usec, L: 0
0923 17:31:30.655199 (+ 10678us) write_op.cc:270] Start()
0923 17:31:30.655323 (+ 124us) write_op.cc:276] Timestamp: P: 1790184677661086 usec, L: 0
0923 17:31:30.677010 (+ 21687us) write_op.cc:270] Start()
0923 17:31:30.677131 (+ 121us) write_op.cc:276] Timestamp: P: 1790184677757426 usec, L: 0
0923 17:31:30.678407 (+ 1276us) write_op.cc:270] Start()
0923 17:31:30.678504 (+ 97us) write_op.cc:276] Timestamp: P: 1790184677996818 usec, L: 0
0923 17:31:30.679027 (+ 523us) write_op.cc:270] Start()
0923 17:31:30.679113 (+ 86us) write_op.cc:276] Timestamp: P: 1790184678016116 usec, L: 0
0923 17:31:30.679551 (+ 438us) write_op.cc:270] Start()
0923 17:31:30.679625 (+ 74us) write_op.cc:276] Timestamp: P: 1790184678153024 usec, L: 0
0923 17:31:30.690661 (+ 11036us) write_op.cc:270] Start()
0923 17:31:30.690831 (+ 170us) write_op.cc:276] Timestamp: P: 1790184678215272 usec, L: 0
0923 17:31:30.691562 (+ 731us) write_op.cc:270] Start()
0923 17:31:30.691644 (+ 82us) write_op.cc:276] Timestamp: P: 1790184678231990 usec, L: 0
0923 17:31:30.692419 (+ 775us) write_op.cc:270] Start()
0923 17:31:30.692501 (+ 82us) write_op.cc:276] Timestamp: P: 1790184678248531 usec, L: 0
0923 17:31:30.693033 (+ 532us) write_op.cc:270] Start()
0923 17:31:30.693122 (+ 89us) write_op.cc:276] Timestamp: P: 1790184678391950 usec, L: 0
0923 17:31:30.693721 (+ 599us) write_op.cc:270] Start()
0923 17:31:30.693796 (+ 75us) write_op.cc:276] Timestamp: P: 1790184678427156 usec, L: 0
0923 17:31:30.694345 (+ 549us) write_op.cc:270] Start()
0923 17:31:30.694479 (+ 134us) write_op.cc:276] Timestamp: P: 1790184678515631 usec, L: 0
0923 17:31:30.695174 (+ 695us) write_op.cc:270] Start()
0923 17:31:30.695250 (+ 76us) write_op.cc:276] Timestamp: P: 1790184678630455 usec, L: 0
0923 17:31:30.695678 (+ 428us) write_op.cc:270] Start()
0923 17:31:30.695758 (+ 80us) write_op.cc:276] Timestamp: P: 1790184678663764 usec, L: 0
0923 17:31:30.737118 (+ 41360us) write_op.cc:270] Start()
0923 17:31:30.737241 (+ 123us) write_op.cc:276] Timestamp: P: 1790184678736473 usec, L: 0
0923 17:31:30.741111 (+ 3870us) write_op.cc:270] Start()
0923 17:31:30.741212 (+ 101us) write_op.cc:276] Timestamp: P: 1790184679046512 usec, L: 0
0923 17:31:30.752585 (+ 11373us) write_op.cc:270] Start()
0923 17:31:30.752702 (+ 117us) write_op.cc:276] Timestamp: P: 1790184679079408 usec, L: 0
0923 17:31:30.753457 (+ 755us) write_op.cc:270] Start()
0923 17:31:30.753532 (+ 75us) write_op.cc:276] Timestamp: P: 1790184679113899 usec, L: 0
0923 17:31:30.753952 (+ 420us) write_op.cc:270] Start()
0923 17:31:30.754033 (+ 81us) write_op.cc:276] Timestamp: P: 1790184679222426 usec, L: 0
0923 17:31:30.754427 (+ 394us) write_op.cc:270] St
I20260923 17:31:32.912803 17027 raft_consensus.cc:2454] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:32.950569 18250 tablet.cc:1722] T 7dbd8eb0ab9243bd81c3d6a893f18aea P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:32.960592 17146 tablet_bootstrap.cc:654] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:32.966806 18095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1dbd0d3324414f26ab5d34c375921ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } } }
I20260923 17:31:32.969767 18095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2 [sys.catalog]: This master's current role is: LEADER
I20260923 17:31:33.003556 17144 tablet_bootstrap.cc:492] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:33.013733 16678 consensus_peers.cc:487] T 8e7c8e360d954ce78913e78c8662c341 P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e7c8e360d954ce78913e78c8662c341. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:33.022161 17146 tablet_bootstrap.cc:492] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:33.022564 17146 ts_tablet_manager.cc:1397] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.109s user 0.008s sys 0.011s
I20260923 17:31:33.025751 17146 raft_consensus.cc:348] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.031258 17146 raft_consensus.cc:374] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.033507 17146 raft_consensus.cc:729] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:33.039063 17146 consensus_queue.cc:260] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.054098 17146 ts_tablet_manager.cc:1428] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.031s user 0.010s sys 0.000s
I20260923 17:31:33.059834 17146 tablet_bootstrap.cc:492] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:33.063747 17144 tablet_bootstrap.cc:654] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.091902 16678 leader_election.cc:302] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:33.094120 18245 raft_consensus.cc:2781] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:33.101778 18245 raft_consensus.cc:686] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:33.102800 18245 consensus_queue.cc:237] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.132786 18204 raft_consensus.cc:504] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.135026 18204 leader_election.cc:288] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:33.143222 17143 tablet_bootstrap.cc:654] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.163765 16678 leader_election.cc:302] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:33.165428 18211 raft_consensus.cc:2781] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:33.165843 18211 raft_consensus.cc:482] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:33.166181 18211 raft_consensus.cc:3037] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:33.178311 18190 raft_consensus.cc:482] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.179211 18190 raft_consensus.cc:504] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.182555 18190 leader_election.cc:288] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:33.199652 17146 tablet_bootstrap.cc:654] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:33.232601 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 67813us
I20260923 17:31:33.235991 17022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:33.237020 17022 raft_consensus.cc:3037] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:33.250144 17022 raft_consensus.cc:2454] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:33.253723 16893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:33.248634 18241 raft_consensus.cc:482] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.254733 18246 raft_consensus.cc:482] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.267141 18167 raft_consensus.cc:482] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.280385 18167 raft_consensus.cc:504] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.286653 16888 raft_consensus.cc:1226] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [term 2 LEADER]: Rejecting Update request from peer be93fda0843b44f5abc49e6f35c69217 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260923 17:31:33.300942 18250 tablet.cc:1722] T a2e5bc4ae186404c896814b95d6cc7ef P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:33.327587 17143 tablet_bootstrap.cc:492] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:33.328190 17143 ts_tablet_manager.cc:1397] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.492s user 0.019s sys 0.003s
I20260923 17:31:33.331352 18289 tablet.cc:1722] T 30d5c465348d4e68a5047d764bb802b5 P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:33.286067 17040 raft_consensus.cc:1261] T a84e322a4de24cbabf450fceb098d6e7 P e5c071cdf65042a59cad226a15a5532c [term 2 FOLLOWER]: Refusing update from remote peer c36e79694e154f84a46980719217d7f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260923 17:31:33.330246 17146 tablet_bootstrap.cc:492] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:33.276687 18246 raft_consensus.cc:504] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.363456 18246 leader_election.cc:288] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:33.271616 18241 raft_consensus.cc:504] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.364020 16886 raft_consensus.cc:3037] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:33.368294 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:33.336877 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e20fc94eecf4a14840ca88177070e33" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:33.369107 18222 ts_tablet_manager.cc:1918] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:33.369508 18222 log.cc:1196] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/261d1d03290546969a74ca60beb890eb
I20260923 17:31:33.369532 18241 leader_election.cc:288] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:33.371610 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:33.372839 16881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:33.379763 17907 ts_manager.cc:194] Registered new tserver with Master: c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:33.391206 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:33.394930 18222 ts_tablet_manager.cc:1939] T 261d1d03290546969a74ca60beb890eb P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:33.398164 18221 ts_tablet_manager.cc:1918] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:33.361620 18167 leader_election.cc:288] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:33.404237 18221 log.cc:1196] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/547fda6e7bcf474e9f73ba9a6659d34f
I20260923 17:31:33.405903 18221 ts_tablet_manager.cc:1939] T 547fda6e7bcf474e9f73ba9a6659d34f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:33.418033 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 278) took 1664 ms. Trace:
I20260923 17:31:33.385005 16792 heartbeater.cc:464] Registering TS with master...
I20260923 17:31:33.371146 18211 raft_consensus.cc:504] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.432617 18236 catalog_manager.cc:1259] Loaded cluster ID: b555dcd485a344bc8bb1e61ebbffcc17
I20260923 17:31:33.432865 18236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: loading cluster ID for follower catalog manager: success
I20260923 17:31:33.435144 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 229) took 1931 ms. Trace:
I20260923 17:31:33.379582 17144 tablet_bootstrap.cc:492] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:33.432649 18222 rpcz_store.cc:276] 0923 17:31:31.753642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.753778 (+ 136us) service_pool.cc:227] Handling call
0923 17:31:32.560000 (+806222us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:32.560043 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.037388 (+477345us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.368812 (+331424us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.417959 (+ 49147us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":185228,"tablet-delete.queue_time_us":805064}
W20260923 17:31:33.357519 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 89582us
I20260923 17:31:33.400671 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e20fc94eecf4a14840ca88177070e33" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:33.454285 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:33.454461 16826 leader_election.cc:302] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:33.455215 18167 raft_consensus.cc:2726] T a71e9d28008c4670b3d48f32161d6aee P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:33.456410 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 261d1d03290546969a74ca60beb890eb (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
I20260923 17:31:33.457557 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 279) took 1699 ms. Trace:
I20260923 17:31:33.457888 18222 rpcz_store.cc:276] 0923 17:31:31.758040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.758175 (+ 135us) service_pool.cc:227] Handling call
0923 17:31:33.457222 (+1699047us) ts_tablet_manager.cc:1145] Deleting tablet 051f2e478a324df98647d90107d55533
0923 17:31:33.457283 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.457494 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1698093}
I20260923 17:31:33.458426 17031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
W20260923 17:31:33.459136 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 051f2e478a324df98647d90107d55533 because the tablet was not found. No further retry: Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:33.434108 18211 leader_election.cc:288] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:33.440588 17144 ts_tablet_manager.cc:1397] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.437s user 0.009s sys 0.005s
I20260923 17:31:33.460588 18255 raft_consensus.cc:482] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.461047 18255 raft_consensus.cc:504] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.455610 17036 raft_consensus.cc:2454] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:33.462234 17144 raft_consensus.cc:348] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.462883 17144 raft_consensus.cc:374] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.463066 18255 leader_election.cc:288] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:33.463292 17144 raft_consensus.cc:729] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:33.463853 17144 consensus_queue.cc:260] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:33.450590 16677 leader_election.cc:341] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:33.465302 16677 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.465577 18167 consensus_queue.cc:1035] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:33.332639 18285 tablet.cc:1722] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:33.350922 18296 tablet.cc:1722] T 7dbd8eb0ab9243bd81c3d6a893f18aea P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:33.340108 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:33.471477 17025 raft_consensus.cc:3037] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:33.472882 16827 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:33.473086 17146 ts_tablet_manager.cc:1397] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.415s user 0.009s sys 0.003s
W20260923 17:31:33.474443 16677 consensus_peers.cc:487] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:33.479802 18242 consensus_queue.cc:1035] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:33.481768 16677 leader_election.cc:341] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.435487 18221 rpcz_store.cc:276] 0923 17:31:31.503871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.506756 (+ 2885us) service_pool.cc:227] Handling call
0923 17:31:32.785370 (+1278614us) ts_tablet_manager.cc:1145] Deleting tablet 547fda6e7bcf474e9f73ba9a6659d34f
0923 17:31:32.785419 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.142253 (+356834us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.397906 (+255653us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.435071 (+ 37165us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":175,"tablet-delete.queue_time_us":1257514}
I20260923 17:31:33.504213 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 547fda6e7bcf474e9f73ba9a6659d34f (table default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]) successfully deleted
W20260923 17:31:33.455533 16678 consensus_peers.cc:487] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 -> Peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Couldn't send request to peer e5c071cdf65042a59cad226a15a5532c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:33.516057 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 60622.3us
I20260923 17:31:33.518767 17023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:33.519546 17023 raft_consensus.cc:3037] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:33.493652 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 230) took 1988 ms. Trace:
I20260923 17:31:33.525620 18221 rpcz_store.cc:276] 0923 17:31:31.505150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.507348 (+ 2198us) service_pool.cc:227] Handling call
0923 17:31:33.492426 (+1985078us) ts_tablet_manager.cc:1145] Deleting tablet a60bf118082b4e9991bfeb3ac59eb118
0923 17:31:33.492480 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.493553 (+ 1073us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1962121}
I20260923 17:31:33.529443 17144 ts_tablet_manager.cc:1428] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.070s user 0.003s sys 0.002s
I20260923 17:31:33.347810 17143 raft_consensus.cc:348] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.534657 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd483bcad624c9a86503bdf15b9d6f8" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:33.535058 17143 raft_consensus.cc:374] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.535346 17143 raft_consensus.cc:729] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
W20260923 17:31:33.539458 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 55112.1us
I20260923 17:31:33.551534 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 231) took 2046 ms. Trace:
W20260923 17:31:33.552558 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet a60bf118082b4e9991bfeb3ac59eb118 because the tablet was not found. No further retry: Not found: Tablet not found: a60bf118082b4e9991bfeb3ac59eb118
I20260923 17:31:33.553428 18303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dbd0d3324414f26ab5d34c375921ef2)
I20260923 17:31:33.555610 18221 rpcz_store.cc:276] 0923 17:31:31.505531 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.508111 (+ 2580us) service_pool.cc:227] Handling call
0923 17:31:33.550820 (+2042709us) ts_tablet_manager.cc:1145] Deleting tablet df53a9825e5348189485f30b8fdedc63
0923 17:31:33.550907 (+ 87us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.551432 (+ 525us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2020079}
W20260923 17:31:33.557245 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet df53a9825e5348189485f30b8fdedc63 because the tablet was not found. No further retry: Not found: Tablet not found: df53a9825e5348189485f30b8fdedc63
I20260923 17:31:33.565655 18242 consensus_queue.cc:1035] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:33.531258 17025 raft_consensus.cc:2454] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:33.567613 18219 ts_tablet_manager.cc:1918] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:33.568084 18219 log.cc:1196] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/006895667f324d049debbebb132ed09b
I20260923 17:31:33.551355 17143 consensus_queue.cc:260] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.569679 16678 leader_election.cc:302] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:33.570282 18255 raft_consensus.cc:482] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.570997 18219 ts_tablet_manager.cc:1939] T 006895667f324d049debbebb132ed09b P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:33.571106 18255 raft_consensus.cc:504] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.573493 18255 leader_election.cc:288] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:33.583321 18245 raft_consensus.cc:2781] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:33.583086 17023 raft_consensus.cc:2454] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:33.584347 18245 raft_consensus.cc:686] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:33.585469 18245 consensus_queue.cc:237] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.586144 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 229) took 1900 ms. Trace:
I20260923 17:31:33.586562 18219 rpcz_store.cc:276] 0923 17:31:31.685417 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.685568 (+ 151us) service_pool.cc:227] Handling call
0923 17:31:32.541420 (+855852us) ts_tablet_manager.cc:1145] Deleting tablet 006895667f324d049debbebb132ed09b
0923 17:31:32.541470 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.206724 (+665254us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.567372 (+360648us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:33.586073 (+ 18701us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":845361}
I20260923 17:31:33.588483 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 230) took 1902 ms. Trace:
I20260923 17:31:33.588955 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 006895667f324d049debbebb132ed09b (table default.Table_516ec99807654f5385fc92422c51562a [id=86f987f92c4947d78061f19b4643121b]) successfully deleted
I20260923 17:31:33.588855 18219 rpcz_store.cc:276] 0923 17:31:31.686484 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.695481 (+ 8997us) service_pool.cc:227] Handling call
0923 17:31:33.588195 (+1892714us) ts_tablet_manager.cc:1145] Deleting tablet df53a9825e5348189485f30b8fdedc63
0923 17:31:33.588253 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.588431 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1891825}
I20260923 17:31:33.589846 18219 tablet_replica.cc:330] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:33.590214 17143 ts_tablet_manager.cc:1428] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.249s user 0.003s sys 0.003s
I20260923 17:31:33.590521 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 232) took 2084 ms. Trace:
W20260923 17:31:33.598188 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet df53a9825e5348189485f30b8fdedc63 because the tablet was not found. No further retry: Not found: Tablet not found: df53a9825e5348189485f30b8fdedc63
I20260923 17:31:33.596894 17143 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:33.596904 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 0f763a4bc57144199b4a4b7b17f12762 (DEFAULT_TABLE table=default.Table_9394ecdeb8cf4220b38b8bf6b6a51bc4 [id=8837ec4e98fe40698cbc56d398a325a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.602203 16712 tablet_service.cc:1467] Processing CreateTablet for tablet daaf7085dc914c6fb924e8497d509c29 (DEFAULT_TABLE table=default.Table_5f3593d4362a4d0d8797c69b90733b9c [id=317ba6b02a6f492f97d3a6e8918b771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.591837 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 0f763a4bc57144199b4a4b7b17f12762 (DEFAULT_TABLE table=default.Table_9394ecdeb8cf4220b38b8bf6b6a51bc4 [id=8837ec4e98fe40698cbc56d398a325a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.599222 18221 rpcz_store.cc:276] 0923 17:31:31.506049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.508703 (+ 2654us) service_pool.cc:227] Handling call
0923 17:31:33.556808 (+2048105us) ts_tablet_manager.cc:1145] Deleting tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d
0923 17:31:33.575893 (+ 19085us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.590454 (+ 14561us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2025828}
I20260923 17:31:33.608717 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f763a4bc57144199b4a4b7b17f12762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.607219 16714 tablet_service.cc:1467] Processing CreateTablet for tablet afe0af54e4e743adbb66434619b1d6c3 (DEFAULT_TABLE table=default.Table_3759087d3e8a4ccd91f2b0436ffdc062 [id=de679a64041d4662bb93e6db868b3df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.593247 17008 tablet_service.cc:1467] Processing CreateTablet for tablet daaf7085dc914c6fb924e8497d509c29 (DEFAULT_TABLE table=default.Table_5f3593d4362a4d0d8797c69b90733b9c [id=317ba6b02a6f492f97d3a6e8918b771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.614678 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0b13a93d4445bebd43690ef36452dd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:33.615218 16886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0b13a93d4445bebd43690ef36452dd" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:33.603477 17146 raft_consensus.cc:348] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.604465 17015 tablet_service.cc:1467] Processing CreateTablet for tablet afe0af54e4e743adbb66434619b1d6c3 (DEFAULT_TABLE table=default.Table_3759087d3e8a4ccd91f2b0436ffdc062 [id=de679a64041d4662bb93e6db868b3df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.615722 17146 raft_consensus.cc:374] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.616140 17146 raft_consensus.cc:729] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:33.616789 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070d38df3f0f43edab3826835c9503ce" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:33.616914 17146 consensus_queue.cc:260] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.617503 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070d38df3f0f43edab3826835c9503ce" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:33.618571 17146 ts_tablet_manager.cc:1428] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.144s user 0.005s sys 0.000s
W20260923 17:31:33.621152 16972 leader_election.cc:341] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.621328 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daaf7085dc914c6fb924e8497d509c29. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:33.621775 16972 leader_election.cc:341] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:33.622488 16972 leader_election.cc:341] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.622804 16972 leader_election.cc:302] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
W20260923 17:31:33.623749 16972 leader_election.cc:341] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.624153 18248 raft_consensus.cc:2726] T 6f0b13a93d4445bebd43690ef36452dd P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:33.625682 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe0af54e4e743adbb66434619b1d6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.630331 17146 tablet_bootstrap.cc:492] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:33.643620 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f9f3ed1bd50d4e2eb30e1a6b6ee8178d because the tablet was not found. No further retry: Not found: Tablet not found: f9f3ed1bd50d4e2eb30e1a6b6ee8178d
I20260923 17:31:33.645479 16866 tablet_service.cc:1467] Processing CreateTablet for tablet 0f763a4bc57144199b4a4b7b17f12762 (DEFAULT_TABLE table=default.Table_9394ecdeb8cf4220b38b8bf6b6a51bc4 [id=8837ec4e98fe40698cbc56d398a325a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.645772 16869 tablet_service.cc:1467] Processing CreateTablet for tablet daaf7085dc914c6fb924e8497d509c29 (DEFAULT_TABLE table=default.Table_5f3593d4362a4d0d8797c69b90733b9c [id=317ba6b02a6f492f97d3a6e8918b771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.646560 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f763a4bc57144199b4a4b7b17f12762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.647001 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daaf7085dc914c6fb924e8497d509c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.620278 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:33.634523 18303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd0d3324414f26ab5d34c375921ef2" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 33707 } } peers { permanent_uuid: "70172383e8bf40a49974968e8a8f5b23" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 45151 } } peers { permanent_uuid: "012948ec54c1441381b43989690264ce" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 46075 } }
I20260923 17:31:33.659400 16399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 309 } } ignore_live_leader: false dest_uuid: "1dbd0d3324414f26ab5d34c375921ef2" is_pre_election: true
I20260923 17:31:33.662109 17143 tablet_bootstrap.cc:654] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.662926 18303 leader_election.cc:288] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dbd0d3324414f26ab5d34c375921ef2 (127.15.246.190:33707), 70172383e8bf40a49974968e8a8f5b23 (127.15.246.189:45151)
I20260923 17:31:33.671015 16713 tablet_service.cc:1467] Processing CreateTablet for tablet 5c83942d30a543d692641024253078cc (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.660146 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 3 got 2
I20260923 17:31:33.674245 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:33.675477 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 5c83942d30a543d692641024253078cc (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260923 17:31:33.675599 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 67249.5us
I20260923 17:31:33.676499 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c83942d30a543d692641024253078cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.678900 16865 tablet_service.cc:1467] Processing CreateTablet for tablet afe0af54e4e743adbb66434619b1d6c3 (DEFAULT_TABLE table=default.Table_3759087d3e8a4ccd91f2b0436ffdc062 [id=de679a64041d4662bb93e6db868b3df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.679742 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe0af54e4e743adbb66434619b1d6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.684397 17012 tablet_service.cc:1467] Processing CreateTablet for tablet 5c83942d30a543d692641024253078cc (DEFAULT_TABLE table=default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.686393 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc89f029c1c4d8fb1b192017cade4f7 (DEFAULT_TABLE table=default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.687592 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc89f029c1c4d8fb1b192017cade4f7 (DEFAULT_TABLE table=default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.687973 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc89f029c1c4d8fb1b192017cade4f7 (DEFAULT_TABLE table=default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:33.688915 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc89f029c1c4d8fb1b192017cade4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.690222 17144 tablet_bootstrap.cc:492] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:33.695355 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f763a4bc57144199b4a4b7b17f12762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.701058 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daaf7085dc914c6fb924e8497d509c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.704973 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe0af54e4e743adbb66434619b1d6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.706395 16713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c83942d30a543d692641024253078cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.710027 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:33.710755 18222 tablet_replica.cc:330] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:33.711289 16380 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:33.712546 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc89f029c1c4d8fb1b192017cade4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.714655 18219 raft_consensus.cc:2229] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:33.715605 18219 raft_consensus.cc:2258] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:33.718873 18219 ts_tablet_manager.cc:1905] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:33.724934 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "012948ec54c1441381b43989690264ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 309 } } ignore_live_leader: false dest_uuid: "70172383e8bf40a49974968e8a8f5b23" is_pre_election: true
I20260923 17:31:33.726616 17830 leader_election.cc:302] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012948ec54c1441381b43989690264ce; no voters: 1dbd0d3324414f26ab5d34c375921ef2, 70172383e8bf40a49974968e8a8f5b23
I20260923 17:31:33.732412 18303 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260923 17:31:33.780313 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 64622.3us
I20260923 17:31:33.788549 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 234) took 2275 ms. Trace:
I20260923 17:31:33.789153 18221 rpcz_store.cc:276] 0923 17:31:31.512828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.512989 (+ 161us) service_pool.cc:227] Handling call
0923 17:31:33.609197 (+2096208us) ts_tablet_manager.cc:1145] Deleting tablet 051f2e478a324df98647d90107d55533
0923 17:31:33.609250 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.788468 (+179218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2077879}
I20260923 17:31:33.791697 18221 tablet_replica.cc:330] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
W20260923 17:31:33.791905 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 051f2e478a324df98647d90107d55533 because the tablet was not found. No further retry: Not found: Tablet not found: 051f2e478a324df98647d90107d55533
W20260923 17:31:33.803956 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e6d9fad7151a4551ba8f2b0e25525cc7 (table default.Table_18752eef74c04de2852506a9255132a7 [id=fe0119a316ba4a558f8fb5babfe70c06], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e6d9fad7151a4551ba8f2b0e25525cc7 consensus state has no leader
I20260923 17:31:33.624054 16972 leader_election.cc:302] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:33.804737 18255 raft_consensus.cc:2726] T 070d38df3f0f43edab3826835c9503ce P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:33.805873 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 182129us
I20260923 17:31:33.813550 17146 tablet_bootstrap.cc:654] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.823786 17144 tablet_bootstrap.cc:654] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.833428 18311 tablet.cc:1722] T a2e5bc4ae186404c896814b95d6cc7ef P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:33.856405 18221 raft_consensus.cc:2229] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:33.857663 18221 raft_consensus.cc:2258] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:33.865928 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c83942d30a543d692641024253078cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:33.866240 17143 tablet_bootstrap.cc:492] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:33.866686 17143 ts_tablet_manager.cc:1397] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.270s user 0.018s sys 0.017s
I20260923 17:31:33.868752 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc89f029c1c4d8fb1b192017cade4f7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:31:33.869290 18214 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.085s user 0.000s sys 0.004s
I20260923 17:31:33.869376 17143 raft_consensus.cc:348] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.870081 17143 raft_consensus.cc:374] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.870330 17143 raft_consensus.cc:729] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:33.870927 17143 consensus_queue.cc:260] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.871547 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:33.873932 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:33.887180 18236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: acquiring CA information for follower catalog manager: success
I20260923 17:31:33.891827 17143 ts_tablet_manager.cc:1428] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260923 17:31:33.893451 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 3 got 2
I20260923 17:31:33.894760 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:33.908358 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:33.909379 17143 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:33.910645 16382 catalog_manager.cc:5495] T 8a2f1143dcd64b2dbd25e17787096903 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
W20260923 17:31:33.831529 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 50498us
I20260923 17:31:33.923403 16678 leader_election.cc:302] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:33.924703 18012 raft_consensus.cc:2781] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:33.925024 18012 raft_consensus.cc:482] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:33.915493 16382 catalog_manager.cc:5495] T ef890c51e92244f4b97c490f24250679 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:33.925359 18012 raft_consensus.cc:3037] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:33.833179 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 144299us
W20260923 17:31:33.927002 16677 leader_election.cc:341] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:33.927655 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:33.931005 17146 tablet_bootstrap.cc:492] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:33.931416 17146 ts_tablet_manager.cc:1397] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.301s user 0.011s sys 0.002s
I20260923 17:31:33.942500 17146 raft_consensus.cc:348] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.943125 17146 raft_consensus.cc:374] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:33.943339 17146 raft_consensus.cc:729] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:33.944521 16382 catalog_manager.cc:5495] T a71e9d28008c4670b3d48f32161d6aee P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:33.948066 17143 tablet_bootstrap.cc:654] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:33.948697 18250 tablet.cc:1722] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:33.950115 16382 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:33.951239 16382 catalog_manager.cc:5495] T ba0ed72a55d74ab5b8b22fba90848094 P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:33.961225 18246 raft_consensus.cc:482] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:33.961680 18246 raft_consensus.cc:504] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:33.962847 18012 raft_consensus.cc:504] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.963532 18246 leader_election.cc:288] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:33.965703 17027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48bda995b284cf6be7bb6f348ec0fd0" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
W20260923 17:31:33.967087 16827 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:31:33.966553 18222 raft_consensus.cc:2229] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
W20260923 17:31:33.969273 18214 log.cc:924] Time spent T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260923 17:31:33.969563 18222 pending_rounds.cc:62] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Trying to abort 1 pending ops.
I20260923 17:31:33.969821 18222 pending_rounds.cc:68] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596505119752192 op_type: NO_OP noop_request { }
I20260923 17:31:33.970242 18222 raft_consensus.cc:2866] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:33.970570 18222 raft_consensus.cc:2258] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:33.982236 18236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 012948ec54c1441381b43989690264ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:31:33.943866 17146 consensus_queue.cc:260] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:33.993182 17146 ts_tablet_manager.cc:1428] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.062s user 0.004s sys 0.004s
I20260923 17:31:33.994207 17146 tablet_bootstrap.cc:492] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
W20260923 17:31:34.057330 18310 consensus_queue.cc:1159] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "be93fda0843b44f5abc49e6f35c69217" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260923 17:31:34.070168 18222 ts_tablet_manager.cc:1905] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:34.080925 17146 tablet_bootstrap.cc:654] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:34.104288 18255 raft_consensus.cc:482] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.112073 18255 raft_consensus.cc:504] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.114079 18255 leader_election.cc:288] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:34.116605 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54f76e45d254086983cf6ee21484063" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:34.118980 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54f76e45d254086983cf6ee21484063" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:34.121471 16972 leader_election.cc:341] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:34.122166 16972 leader_election.cc:341] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.122484 16972 leader_election.cc:302] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:34.123286 18219 ts_tablet_manager.cc:1918] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:34.123878 18219 log.cc:1196] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:34.124929 18248 raft_consensus.cc:2726] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:34.135648 16870 tablet_service.cc:1467] Processing CreateTablet for tablet e7ae6934972c4b22b3f7e15fe354ac99 (DEFAULT_TABLE table=default.Table_81951dda1cdb4e91a19d5fec01501dff [id=bff454bb71fe41bebc5b960b480720fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:34.136860 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ae6934972c4b22b3f7e15fe354ac99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:34.136803 17146 tablet_bootstrap.cc:492] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:34.138391 17146 ts_tablet_manager.cc:1397] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.144s user 0.012s sys 0.000s
I20260923 17:31:34.138577 18246 raft_consensus.cc:482] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.139429 18246 raft_consensus.cc:504] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:34.140524 17146 raft_consensus.cc:348] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.141105 17146 raft_consensus.cc:374] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.141322 17146 raft_consensus.cc:729] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.144018 17027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:34.147658 18246 leader_election.cc:288] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:33.861112 18221 ts_tablet_manager.cc:1905] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:34.154870 17146 consensus_queue.cc:260] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.163859 17146 ts_tablet_manager.cc:1428] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260923 17:31:34.166523 17010 tablet_service.cc:1467] Processing CreateTablet for tablet e7ae6934972c4b22b3f7e15fe354ac99 (DEFAULT_TABLE table=default.Table_81951dda1cdb4e91a19d5fec01501dff [id=bff454bb71fe41bebc5b960b480720fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:34.167933 16346 master-stress-test.cc:578] Killing test thread 1/10
I20260923 17:31:34.173280 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:34.124954 18219 ts_tablet_manager.cc:1939] T c48bda995b284cf6be7bb6f348ec0fd0 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:34.190541 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:34.200685 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 30d5c465348d4e68a5047d764bb802b5 (table default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8]). Expected version 3 got 2
I20260923 17:31:34.201872 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:34.204963 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e]). Expected version 1 got 0
I20260923 17:31:34.209508 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:34.210336 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:34.211448 16381 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a2e5bc4ae186404c896814b95d6cc7ef (table default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003]). Expected version 1 got 0
I20260923 17:31:34.214843 16386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet db3504813e9c4698a546b6d6fa0fbcb6
I20260923 17:31:34.216182 16386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 246) took 2549 ms. Trace:
I20260923 17:31:34.227928 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:32 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:34.228592 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:32 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.216565 16386 rpcz_store.cc:276] 0923 17:31:31.667238 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.381915 (+714677us) service_pool.cc:227] Handling call
0923 17:31:32.383513 (+ 1598us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:32.417471 (+ 33958us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:32.417607 (+ 136us) catalog_manager.cc:2761] Locking tablets
0923 17:31:32.815882 (+398275us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:32.834497 (+ 18615us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:32.834687 (+ 190us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:34.214731 (+1380044us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:34.216130 (+ 1399us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.817816 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.818230 (+ 414us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.818255 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:32.818271 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:32.818596 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.818637 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.818665 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:32.818946 (+ 281us) tablet.cc:699] Row locks acquired
0923 17:31:32.818961 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:32.819072 (+ 111us) write_op.cc:270] Start()
0923 17:31:32.819166 (+ 94us) write_op.cc:276] Timestamp: P: 1790184692819009 usec, L: 0
0923 17:31:32.819185 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.819710 (+ 525us) log.cc:842] Serialized 799 byte log entry
0923 17:31:32.829671 (+ 9961us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.830017 (+ 346us) write_op.cc:301] APPLY: starting
0923 17:31:32.830075 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.830187 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.830294 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.830564 (+ 270us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.830701 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:32.830738 (+ 37us) write_op.cc:312] APPLY: finished
0923 17:31:32.830945 (+ 207us) log.cc:842] Serialized 29 byte log entry
0923 17:31:32.831096 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.831179 (+ 83us) write_op.cc:454] Released schema lock
0923 17:31:32.831211 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:32.831395 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.831416 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":195,"apply.run_cpu_time_us":1599,"apply.run_wall_time_us":1598,"num_ops":2,"prepare.queue_time_us":156,"prepare.run_cpu_time_us":2199,"prepare.run_wall_time_us":2199,"raft.queue_time_us":1263,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":594,"replication_time_us":10392}]]}
W20260923 17:31:34.233728 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 41) took 1398 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.234299 16386 rpcz_store.cc:269] 0923 17:31:32.835478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.233566 (+1398088us) service_pool.cc:214] Skipping call since client already timed out
0923 17:31:34.233675 (+ 109us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260923 17:31:34.235383 16385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_5fbe014655974cf987e980ebcb49dbc9 [id=2c1a84da38e3492bab350ae4d3e62eb4] alter complete (version 2)
I20260923 17:31:34.247124 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:34.265113 17146 tablet_bootstrap.cc:492] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:34.268772 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.270598 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.271349 18012 leader_election.cc:288] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Requested vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:34.271651 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ae6934972c4b22b3f7e15fe354ac99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:34.272800 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:34.276806 16383 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 42) took 5967 ms (client timeout 1000 ms). Trace:
I20260923 17:31:34.286108 17146 tablet_bootstrap.cc:654] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:34.287901 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet 547fda6e7bcf474e9f73ba9a6659d34f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:34.288658 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:34.289549 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 38) took 5551 ms (client timeout 1000 ms). Trace:
I20260923 17:31:34.285864 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 236) took 2085 ms. Trace:
I20260923 17:31:34.293493 18219 rpcz_store.cc:276] 0923 17:31:32.200847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.211926 (+ 11079us) service_pool.cc:227] Handling call
0923 17:31:33.589702 (+1377776us) ts_tablet_manager.cc:1145] Deleting tablet c48bda995b284cf6be7bb6f348ec0fd0
0923 17:31:33.589738 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.887565 (+297827us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.122900 (+235335us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.285767 (+162867us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":102330,"tablet-delete.queue_time_us":1377124}
W20260923 17:31:34.299961 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 372016us
W20260923 17:31:34.303342 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 378867us
W20260923 17:31:34.308926 16388 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c consensus state has no leader
I20260923 17:31:34.318485 17143 tablet_bootstrap.cc:492] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:34.318966 17143 ts_tablet_manager.cc:1397] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.410s user 0.050s sys 0.051s
W20260923 17:31:34.277561 16383 rpcz_store.cc:269] 0923 17:31:28.309339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.461504 (+152165us) service_pool.cc:227] Handling call
0923 17:31:30.605098 (+2143594us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:30.605128 (+ 30us) write_op.cc:454] Released schema lock
0923 17:31:34.276737 (+3671609us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:30.573999 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:30.574966 (+ 967us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:30.575003 (+ 37us) write_op.cc:435] Acquired schema lock
0923 17:31:30.575053 (+ 50us) tablet.cc:660] Decoding operations
0923 17:31:30.575812 (+ 759us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:30.575861 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:30.575889 (+ 28us) tablet.cc:683] Acquiring locks for 3 operations
0923 17:31:30.576717 (+ 828us) tablet.cc:699] Row locks acquired
0923 17:31:30.576742 (+ 25us) write_op.cc:260] PREPARE: finished
0923 17:31:30.576886 (+ 144us) write_op.cc:270] Start()
0923 17:31:30.576993 (+ 107us) write_op.cc:276] Timestamp: P: 1790184690576816 usec, L: 0
0923 17:31:30.577015 (+ 22us) op_driver.cc:331] REPLICATION: starting
0923 17:31:30.577718 (+ 703us) log.cc:842] Serialized 1302 byte log entry
0923 17:31:30.590388 (+ 12670us) op_driver.cc:444] REPLICATION: finished
0923 17:31:30.590789 (+ 401us) write_op.cc:301] APPLY: starting
0923 17:31:30.590860 (+ 71us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:30.591099 (+ 239us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:30.591197 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:30.591793 (+ 596us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:30.592179 (+ 386us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0923 17:31:30.594154 (+ 1975us) write_op.cc:312] APPLY: finished
0923 17:31:30.594418 (+ 264us) log.cc:842] Serialized 35 byte log entry
0923 17:31:30.594598 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:30.594727 (+ 129us) write_op.cc:454] Released schema lock
0923 17:31:30.594792 (+ 65us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0923 17:31:31.588879 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.589637 (+ 758us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.589702 (+ 65us) write_op.cc:435] Acquired schema lock
0923 17:31:31.589754 (+ 52us) tablet.cc:660] Decoding operations
0923 17:31:31.590310 (+ 556us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.590414 (+ 104us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.590442 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:31.590704 (+ 262us) tablet.cc:699] Row locks acquired
0923 17:31:31.590719 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:31.590899 (+ 180us) write_op.cc:270] Start()
0923 17:31:31.591033 (+ 134us) write_op.cc:276] Timestamp: P: 1790184691590836 usec, L: 0
0923 17:31:31.591058 (+ 25us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.591951 (+ 893us) log.cc:842] Serialized 356 byte log entry
0923 17:31:31.622370 (+ 30419us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.623096 (+ 726us) write_op.cc:301] APPLY: starting
0923 17:31:31.623143 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.623227 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.623395 (+ 168us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.623573 (+ 178us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.623694 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:31.623745 (+ 51us) write_op.cc:312] APPLY: finished
0923 17:31:31.623988 (+ 243us) log.cc:842] Serialized 23 byte log entry
0923 17:31:31.624178 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.624329 (+ 151us) write_op.cc:454] Released schema lock
0923 17:31:31.624392 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:31.624685 (+ 293us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.624737 (+ 52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":178604,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":228,"apply.run_cpu_time_us":2469,"apply.run_wall_time_us":4312,"num_ops":3,"prepare.queue_time_us":53432,"prepare.run_cpu_time_us":3973,"prepare.run_wall_time_us":14044,"raft.queue_time_us":103,"raft.run_cpu_time_us":670,"raft.run_wall_time_us":670,"replication_time_us":13288}],["op",{"apply.queue_time_us":593,"apply.run_cpu_time_us":1783,"apply.run_wall_time_us":1783,"num_ops":1,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":3567,"prepare.run_wall_time_us":6357,"raft.queue_time_us":16087,"raft.run_cpu_time_us":582,"raft.run_wall_time_us":10967,"replication_time_us":31250}]]}
I20260923 17:31:34.307834 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.322693 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.323427 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c115e29673747f8b0712d441ac993fb with delete_type TABLET_DATA_DELETED (replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:34.289980 16379 rpcz_store.cc:269] 0923 17:31:28.738416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.921468 (+183052us) service_pool.cc:227] Handling call
0923 17:31:29.688576 (+767108us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.688603 (+ 27us) write_op.cc:454] Released schema lock
0923 17:31:34.289483 (+4600880us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:29.657043 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:29.657685 (+ 642us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:29.657711 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:29.657727 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:29.658524 (+ 797us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:29.658586 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:29.658625 (+ 39us) tablet.cc:683] Acquiring locks for 7 operations
0923 17:31:29.659250 (+ 625us) tablet.cc:699] Row locks acquired
0923 17:31:29.659266 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:29.659433 (+ 167us) write_op.cc:270] Start()
0923 17:31:29.659554 (+ 121us) write_op.cc:276] Timestamp: P: 1790184689659345 usec, L: 0
0923 17:31:29.659582 (+ 28us) op_driver.cc:331] REPLICATION: starting
0923 17:31:29.660766 (+ 1184us) log.cc:842] Serialized 2835 byte log entry
0923 17:31:29.675354 (+ 14588us) op_driver.cc:444] REPLICATION: finished
0923 17:31:29.675719 (+ 365us) write_op.cc:301] APPLY: starting
0923 17:31:29.675771 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:29.675935 (+ 164us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:29.676026 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:29.676744 (+ 718us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:29.676904 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0923 17:31:29.676944 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:29.677160 (+ 216us) log.cc:842] Serialized 59 byte log entry
0923 17:31:29.677287 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:29.677385 (+ 98us) write_op.cc:454] Released schema lock
0923 17:31:29.677419 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":51500,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":1869,"apply.run_wall_time_us":1869,"num_ops":7,"prepare.queue_time_us":30716,"prepare.run_cpu_time_us":4129,"prepare.run_wall_time_us":15214,"raft.queue_time_us":77,"raft.run_cpu_time_us":661,"raft.run_wall_time_us":661,"replication_time_us":15664}]]}
I20260923 17:31:34.326150 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet c48bda995b284cf6be7bb6f348ec0fd0 (table default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]) successfully deleted
I20260923 17:31:34.332108 18219 tablet_replica.cc:330] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:34.336071 16678 leader_election.cc:302] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:34.301147 16677 leader_election.cc:341] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.350311 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:34.353380 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 35) took 5753 ms (client timeout 1000 ms). Trace:
I20260923 17:31:34.358449 16712 tablet_service.cc:1467] Processing CreateTablet for tablet e7ae6934972c4b22b3f7e15fe354ac99 (DEFAULT_TABLE table=default.Table_81951dda1cdb4e91a19d5fec01501dff [id=bff454bb71fe41bebc5b960b480720fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:34.359941 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ae6934972c4b22b3f7e15fe354ac99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:34.360914 18316 raft_consensus.cc:2781] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:34.361431 18316 raft_consensus.cc:686] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:34.362372 18316 consensus_queue.cc:237] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:34.312162 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 37) took 3142 ms (client timeout 1000 ms). Trace:
I20260923 17:31:34.365540 16678 leader_election.cc:302] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
W20260923 17:31:34.366051 16388 rpcz_store.cc:269] 0923 17:31:31.169949 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.465705 (+295756us) service_pool.cc:227] Handling call
0923 17:31:32.102850 (+637145us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.102875 (+ 25us) write_op.cc:454] Released schema lock
0923 17:31:34.312091 (+2209216us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.079510 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.079947 (+ 437us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.079973 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:32.079986 (+ 13us) tablet.cc:660] Decoding operations
0923 17:31:32.080267 (+ 281us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.080309 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.080337 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:32.080506 (+ 169us) tablet.cc:699] Row locks acquired
0923 17:31:32.080518 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:32.080625 (+ 107us) write_op.cc:270] Start()
0923 17:31:32.080709 (+ 84us) write_op.cc:276] Timestamp: P: 1790184692080521 usec, L: 0
0923 17:31:32.080723 (+ 14us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.081211 (+ 488us) log.cc:842] Serialized 536 byte log entry
0923 17:31:32.093058 (+ 11847us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.093381 (+ 323us) write_op.cc:301] APPLY: starting
0923 17:31:32.093430 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.093512 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.093603 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.093747 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.093871 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:32.093909 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:32.094066 (+ 157us) log.cc:842] Serialized 23 byte log entry
0923 17:31:32.094197 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.094256 (+ 59us) write_op.cc:454] Released schema lock
0923 17:31:32.094290 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":70394,"reported_tablets":55,"child_traces":[["op",{"apply.queue_time_us":192,"apply.run_cpu_time_us":1076,"apply.run_wall_time_us":1075,"num_ops":1,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":1963,"prepare.run_wall_time_us":1965,"raft.queue_time_us":244,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":603,"replication_time_us":12262}]]}
I20260923 17:31:34.369889 17144 tablet_bootstrap.cc:492] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:34.370399 17144 ts_tablet_manager.cc:1397] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.680s user 0.020s sys 0.000s
I20260923 17:31:34.370663 16388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_3f2ddad73bd74b15a8b9831399e83554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:31:34.372704 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:32 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.373368 17143 raft_consensus.cc:348] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:34.374082 17143 raft_consensus.cc:374] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.374331 17143 raft_consensus.cc:729] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.376784 18313 raft_consensus.cc:2781] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:34.377086 17907 ts_manager.cc:194] Registered new tserver with Master: be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:34.380434 18313 raft_consensus.cc:686] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:34.382484 17143 consensus_queue.cc:260] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
W20260923 17:31:34.386189 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 44921.4us
W20260923 17:31:34.386879 16677 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:34.391883 16677 leader_election.cc:341] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.397063 18219 raft_consensus.cc:2229] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:34.397739 18219 raft_consensus.cc:2258] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:34.404516 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:34.400013 18219 ts_tablet_manager.cc:1905] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:34.415589 16388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f2ddad73bd74b15a8b9831399e83554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:34.377948 16387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 43) took 4818 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.416762 16387 rpcz_store.cc:269] 0923 17:31:29.559419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:29.597443 (+ 38024us) service_pool.cc:227] Handling call
0923 17:31:31.546396 (+1948953us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.546423 (+ 27us) write_op.cc:454] Released schema lock
0923 17:31:34.377880 (+2831457us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:31.493364 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.494344 (+ 980us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.494400 (+ 56us) write_op.cc:435] Acquired schema lock
0923 17:31:31.494453 (+ 53us) tablet.cc:660] Decoding operations
0923 17:31:31.495158 (+ 705us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.495205 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.495230 (+ 25us) tablet.cc:683] Acquiring locks for 4 operations
0923 17:31:31.495852 (+ 622us) tablet.cc:699] Row locks acquired
0923 17:31:31.495897 (+ 45us) write_op.cc:260] PREPARE: finished
0923 17:31:31.496149 (+ 252us) write_op.cc:270] Start()
0923 17:31:31.496317 (+ 168us) write_op.cc:276] Timestamp: P: 1790184691496074 usec, L: 0
0923 17:31:31.496364 (+ 47us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.499124 (+ 2760us) log.cc:842] Serialized 1686 byte log entry
0923 17:31:31.511591 (+ 12467us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.511930 (+ 339us) write_op.cc:301] APPLY: starting
0923 17:31:31.511985 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.512199 (+ 214us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.512390 (+ 191us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.513114 (+ 724us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.513419 (+ 305us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0923 17:31:31.513531 (+ 112us) write_op.cc:312] APPLY: finished
0923 17:31:31.513899 (+ 368us) log.cc:842] Serialized 41 byte log entry
0923 17:31:31.514124 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.514285 (+ 161us) write_op.cc:454] Released schema lock
0923 17:31:31.514348 (+ 63us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":101028,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":2676,"apply.run_wall_time_us":2699,"num_ops":4,"prepare.queue_time_us":5814,"prepare.run_cpu_time_us":4225,"prepare.run_wall_time_us":28403,"raft.queue_time_us":80,"raft.run_cpu_time_us":528,"raft.run_wall_time_us":529,"replication_time_us":15124}]]}
I20260923 17:31:34.421375 16387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_632f010f9d114ecbab7901aa664556ac" } modify_external_catalogs: true
I20260923 17:31:34.428325 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 051f2e478a324df98647d90107d55533 with delete_type TABLET_DATA_DELETED (replaced by tablet 107a073567314be996d6b8e3dbc47701) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.433439 17146 tablet_bootstrap.cc:492] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:34.433885 17146 ts_tablet_manager.cc:1397] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.169s user 0.006s sys 0.009s
I20260923 17:31:34.434352 17143 ts_tablet_manager.cc:1428] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.115s user 0.004s sys 0.006s
I20260923 17:31:34.435431 17143 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:34.381507 18313 consensus_queue.cc:237] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.404323 18250 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42676 (request call id 285) took 1417 ms. Trace:
I20260923 17:31:34.441078 16713 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c929b8beaaf4c7aa668852b63af2204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:28 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.442502 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 006895667f324d049debbebb132ed09b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.436734 18250 rpcz_store.cc:276] 0923 17:31:32.963870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.964069 (+ 199us) service_pool.cc:227] Handling call
0923 17:31:34.381515 (+1417446us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.965428 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0923 17:31:32.965858 (+ 430us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0923 17:31:32.965914 (+ 56us) alter_schema_op.cc:65] Acquired schema lock
0923 17:31:32.966003 (+ 89us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0923 17:31:32.966181 (+ 178us) alter_schema_op.cc:126] START. Timestamp: 1790184692966064
0923 17:31:32.966253 (+ 72us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.967052 (+ 799us) log.cc:842] Serialized 173 byte log entry
0923 17:31:33.928866 (+961814us) op_driver.cc:444] REPLICATION: finished
0923 17:31:33.948512 (+ 19646us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0923 17:31:33.950718 (+ 2206us) log.cc:842] Serialized 5 byte log entry
0923 17:31:34.274167 (+323449us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.380658 (+106491us) log.cc:842] Serialized 14 byte log entry
0923 17:31:34.381344 (+ 686us) alter_schema_op.cc:71] Released schema lock
0923 17:31:34.381371 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10663,"mutex_wait_us":10399,"prepare.queue_time_us":158,"prepare.run_cpu_time_us":7759,"prepare.run_wall_time_us":79540,"raft.queue_time_us":10662,"raft.run_cpu_time_us":742,"raft.run_wall_time_us":1474,"replication_time_us":945433,"thread_start_us":66753,"threads_started":1,"wal-append.queue_time_us":67760}]]}
I20260923 17:31:34.452271 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd with delete_type TABLET_DATA_DELETED (replaced by tablet d15e626ea47f435b8cc75d91d3d04900) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.458240 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7c8e360d954ce78913e78c8662c341 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a992ad5280444dda64c1325da92a707) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.459712 16714 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.463165 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c115e29673747f8b0712d441ac993fb with delete_type TABLET_DATA_DELETED (replaced by tablet bc14a3192ad9429ea31bb7f1a7674ef7) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.464442 16387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_632f010f9d114ecbab7901aa664556ac" } modify_external_catalogs: true
W20260923 17:31:34.396338 16384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 44) took 3725 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.468206 16384 rpcz_store.cc:269] 0923 17:31:30.671050 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.414906 (+743856us) service_pool.cc:227] Handling call
0923 17:31:32.774011 (+1359105us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.774040 (+ 29us) write_op.cc:454] Released schema lock
0923 17:31:34.396255 (+1622215us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:32.746506 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:32.746934 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:32.746965 (+ 31us) write_op.cc:435] Acquired schema lock
0923 17:31:32.746979 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:32.747531 (+ 552us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:32.747574 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:32.747604 (+ 30us) tablet.cc:683] Acquiring locks for 4 operations
0923 17:31:32.747938 (+ 334us) tablet.cc:699] Row locks acquired
0923 17:31:32.747955 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:32.748085 (+ 130us) write_op.cc:270] Start()
0923 17:31:32.748176 (+ 91us) write_op.cc:276] Timestamp: P: 1790184692748023 usec, L: 0
0923 17:31:32.748192 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:32.748897 (+ 705us) log.cc:842] Serialized 1686 byte log entry
0923 17:31:32.757058 (+ 8161us) op_driver.cc:444] REPLICATION: finished
0923 17:31:32.757401 (+ 343us) write_op.cc:301] APPLY: starting
0923 17:31:32.757464 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:32.757607 (+ 143us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:32.757710 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:32.758137 (+ 427us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:32.758295 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0923 17:31:32.758341 (+ 46us) write_op.cc:312] APPLY: finished
0923 17:31:32.758572 (+ 231us) log.cc:842] Serialized 41 byte log entry
0923 17:31:32.758738 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:32.758834 (+ 96us) write_op.cc:454] Released schema lock
0923 17:31:32.758873 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":294485,"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":178,"apply.run_cpu_time_us":1664,"apply.run_wall_time_us":1663,"num_ops":4,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":2701,"prepare.run_wall_time_us":3545,"raft.queue_time_us":98,"raft.run_cpu_time_us":680,"raft.run_wall_time_us":680,"replication_time_us":8769}]]}
W20260923 17:31:34.353826 16385 rpcz_store.cc:269] 0923 17:31:28.600214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:28.773021 (+172807us) service_pool.cc:227] Handling call
0923 17:31:31.290334 (+2517313us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.290359 (+ 25us) write_op.cc:454] Released schema lock
0923 17:31:34.353311 (+3062952us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:31.266574 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.267087 (+ 513us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.267113 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:31.267129 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:31.267560 (+ 431us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.267607 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.267633 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:31.267799 (+ 166us) tablet.cc:699] Row locks acquired
0923 17:31:31.267814 (+ 15us) write_op.cc:260] PREPARE: finished
0923 17:31:31.267970 (+ 156us) write_op.cc:270] Start()
0923 17:31:31.268085 (+ 115us) write_op.cc:276] Timestamp: P: 1790184691267911 usec, L: 0
0923 17:31:31.268104 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.268742 (+ 638us) log.cc:842] Serialized 356 byte log entry
0923 17:31:31.281500 (+ 12758us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.281876 (+ 376us) write_op.cc:301] APPLY: starting
0923 17:31:31.281930 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.282021 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.282139 (+ 118us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.282368 (+ 229us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.282510 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:31.282593 (+ 83us) write_op.cc:312] APPLY: finished
0923 17:31:31.282899 (+ 306us) log.cc:842] Serialized 23 byte log entry
0923 17:31:31.283100 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.283202 (+ 102us) write_op.cc:454] Released schema lock
0923 17:31:31.283256 (+ 54us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0923 17:31:31.669041 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:31.669658 (+ 617us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:31.669688 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:31.669704 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:31.670260 (+ 556us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:31.670328 (+ 68us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:31.670388 (+ 60us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:31.670634 (+ 246us) tablet.cc:699] Row locks acquired
0923 17:31:31.670652 (+ 18us) write_op.cc:260] PREPARE: finished
0923 17:31:31.670768 (+ 116us) write_op.cc:270] Start()
0923 17:31:31.670856 (+ 88us) write_op.cc:276] Timestamp: P: 1790184691670709 usec, L: 0
0923 17:31:31.670878 (+ 22us) op_driver.cc:331] REPLICATION: starting
0923 17:31:31.671463 (+ 585us) log.cc:842] Serialized 356 byte log entry
0923 17:31:31.717632 (+ 46169us) op_driver.cc:444] REPLICATION: finished
0923 17:31:31.717948 (+ 316us) write_op.cc:301] APPLY: starting
0923 17:31:31.718024 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:31.718216 (+ 192us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:31.718432 (+ 216us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:31.718733 (+ 301us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:31.718996 (+ 263us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:31.719093 (+ 97us) write_op.cc:312] APPLY: finished
0923 17:31:31.719346 (+ 253us) log.cc:842] Serialized 23 byte log entry
0923 17:31:31.719485 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.719546 (+ 61us) write_op.cc:454] Released schema lock
0923 17:31:31.719574 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:31.719879 (+ 305us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:31.719932 (+ 53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":135450,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":214,"apply.run_cpu_time_us":1717,"apply.run_wall_time_us":1716,"num_ops":1,"prepare.queue_time_us":164,"prepare.run_cpu_time_us":2502,"prepare.run_wall_time_us":2500,"raft.queue_time_us":2061,"raft.run_cpu_time_us":1075,"raft.run_wall_time_us":1075,"replication_time_us":13315}],["op",{"apply.queue_time_us":117,"apply.run_cpu_time_us":2236,"apply.run_wall_time_us":2235,"num_ops":1,"prepare.queue_time_us":32962,"prepare.run_cpu_time_us":2612,"prepare.run_wall_time_us":3648,"replication_time_us":46684}]]}
I20260923 17:31:34.474887 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_5e18039362c44fc1898d2a728f318340"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:34.476773 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 83995.5us
W20260923 17:31:34.476783 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e18039362c44fc1898d2a728f318340 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:31:34.477802 16677 leader_election.cc:341] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.436045 17146 raft_consensus.cc:348] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.478397 17146 raft_consensus.cc:374] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.478750 17146 raft_consensus.cc:729] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.479507 17146 consensus_queue.cc:260] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.481818 17146 ts_tablet_manager.cc:1428] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
W20260923 17:31:34.424391 16382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 45) took 2040 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.482167 16382 rpcz_store.cc:269] 0923 17:31:32.383608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.498347 (+114739us) service_pool.cc:227] Handling call
0923 17:31:34.424328 (+1925981us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:33.995184 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:33.996112 (+ 928us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:33.996166 (+ 54us) write_op.cc:435] Acquired schema lock
0923 17:31:33.996210 (+ 44us) tablet.cc:660] Decoding operations
0923 17:31:33.997239 (+ 1029us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:33.997373 (+ 134us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:33.997430 (+ 57us) tablet.cc:683] Acquiring locks for 4 operations
0923 17:31:33.998062 (+ 632us) tablet.cc:699] Row locks acquired
0923 17:31:33.998079 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:33.998202 (+ 123us) write_op.cc:270] Start()
0923 17:31:33.998283 (+ 81us) write_op.cc:276] Timestamp: P: 1790184693998161 usec, L: 0
0923 17:31:33.998301 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:33.998837 (+ 536us) log.cc:842] Serialized 1686 byte log entry
0923 17:31:34.040976 (+ 42139us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.041212 (+ 236us) write_op.cc:301] APPLY: starting
0923 17:31:34.041265 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.041400 (+ 135us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.041488 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.041853 (+ 365us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.041995 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0923 17:31:34.042033 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:34.042219 (+ 186us) log.cc:842] Serialized 41 byte log entry
0923 17:31:34.042363 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.042444 (+ 81us) write_op.cc:454] Released schema lock
0923 17:31:34.042473 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:34.042637 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.042660 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":86126,"reported_tablets":57,"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":1611,"apply.run_wall_time_us":1610,"num_ops":4,"prepare.queue_time_us":27103,"prepare.run_cpu_time_us":4292,"prepare.run_wall_time_us":30599,"raft.queue_time_us":232,"raft.run_cpu_time_us":1957,"raft.run_wall_time_us":1956,"replication_time_us":42595,"thread_start_us":32140,"threads_started":1}]]}
W20260923 17:31:34.407469 16380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 40) took 3074 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.485670 16380 rpcz_store.cc:269] 0923 17:31:31.332785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.611747 (+278962us) service_pool.cc:227] Handling call
0923 17:31:34.407393 (+2795646us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":48}
I20260923 17:31:34.488409 16380 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_b0f775b6c72d4f55bfeba14c7a8c00e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:34.490123 16380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0f775b6c72d4f55bfeba14c7a8c00e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:34.491951 17144 raft_consensus.cc:348] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.495360 17144 raft_consensus.cc:374] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.495692 17144 raft_consensus.cc:729] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.496425 17144 consensus_queue.cc:260] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.498845 16384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_ed9555c29504496aac9478914859831b" } new_table_name: "default.Table_c1bfc0841ebe4015841fdddeeae9b6e9" modify_external_catalogs: true
I20260923 17:31:34.512861 17143 tablet_bootstrap.cc:654] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:34.522511 18222 ts_tablet_manager.cc:1918] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:34.522953 18222 log.cc:1196] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:34.523649 18222 ts_tablet_manager.cc:1939] T c48bda995b284cf6be7bb6f348ec0fd0 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
W20260923 17:31:34.530229 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 51621.1us
W20260923 17:31:34.530979 16677 leader_election.cc:341] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.554621 16382 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48560:
table { table_name: "default.Table_dbbb33fb20264319a1ace8d6d1553144" } new_table_name: "default.Table_3f9064b9a5b146cd89d2ec54efa1fa4b" modify_external_catalogs: true
I20260923 17:31:34.561188 17023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e20fc94eecf4a14840ca88177070e33" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c"
I20260923 17:31:34.564913 17023 raft_consensus.cc:3037] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:34.575767 17144 ts_tablet_manager.cc:1428] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.205s user 0.006s sys 0.000s
I20260923 17:31:34.582393 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 280) took 2297 ms. Trace:
I20260923 17:31:34.582772 18222 rpcz_store.cc:276] 0923 17:31:32.285106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.287071 (+ 1965us) service_pool.cc:227] Handling call
0923 17:31:33.536248 (+1249177us) ts_tablet_manager.cc:1145] Deleting tablet c48bda995b284cf6be7bb6f348ec0fd0
0923 17:31:33.694631 (+158383us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:33.969036 (+274405us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400123ed4. stack: 00007f24a4f0c033 00007f24a4f0bf30 00007f24a4a8d534 00000000004db2b4 00000000004db249 00007f24abe18471 00007f24abe9b352 00007f24ac2b126c 00007f24acebecfa 00007f24aceeee10 00007f24aced5e0d 00007f24aced5db9 00007f24aced5d49 00007f24aced5d11 00007f24aced4f3d 0000000000580784
0923 17:31:34.428166 (+459130us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.522246 (+ 94080us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.582307 (+ 60061us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":62828,"spinlock_wait_cycles":536719104,"tablet-delete.queue_time_us":1248110}
I20260923 17:31:34.585256 17144 tablet_bootstrap.cc:492] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:34.588423 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet c48bda995b284cf6be7bb6f348ec0fd0 (table default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]) successfully deleted
I20260923 17:31:34.599007 17146 tablet_bootstrap.cc:492] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:34.601364 17144 tablet_bootstrap.cc:654] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:34.587721 18222 tablet_replica.cc:330] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:34.616778 18219 ts_tablet_manager.cc:1918] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:34.631222 18219 log.cc:1196] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/456273cf57314cb8a05c8e765543913f
I20260923 17:31:34.627400 17146 tablet_bootstrap.cc:654] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:34.645090 18219 ts_tablet_manager.cc:1939] T 456273cf57314cb8a05c8e765543913f P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:34.628408 18255 raft_consensus.cc:482] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.647948 18255 raft_consensus.cc:504] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.649621 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 237) took 2445 ms. Trace:
I20260923 17:31:34.649973 18255 leader_election.cc:288] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:34.650800 17023 raft_consensus.cc:2454] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 1.
I20260923 17:31:34.651392 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8fc3755f2645289656aef924ac6f38" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:34.653261 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8fc3755f2645289656aef924ac6f38" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:34.653659 16972 leader_election.cc:341] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:34.655920 16972 leader_election.cc:341] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.656256 16972 leader_election.cc:302] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:34.657593 18255 raft_consensus.cc:2726] T 1a8fc3755f2645289656aef924ac6f38 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:34.626261 18332 tablet.cc:1722] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260923 17:31:34.643119 18337 raft_consensus.cc:482] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.658788 18337 raft_consensus.cc:504] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.661724 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b4d39a636e44e68ef2229006551ffa" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:34.663197 16972 leader_election.cc:341] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.664183 18337 leader_election.cc:288] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:34.665297 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b4d39a636e44e68ef2229006551ffa" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:34.666563 16972 leader_election.cc:341] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:34.667090 16972 leader_election.cc:302] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:34.668021 18337 raft_consensus.cc:2726] T b7b4d39a636e44e68ef2229006551ffa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:34.649967 18219 rpcz_store.cc:276] 0923 17:31:32.204009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.212636 (+ 8627us) service_pool.cc:227] Handling call
0923 17:31:34.331845 (+2119209us) ts_tablet_manager.cc:1145] Deleting tablet 456273cf57314cb8a05c8e765543913f
0923 17:31:34.331898 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:34.486953 (+155055us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.616494 (+129541us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:34.649551 (+ 33057us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63720,"tablet-delete.queue_time_us":2118675}
I20260923 17:31:34.712670 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e20fc94eecf4a14840ca88177070e33" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2"
I20260923 17:31:34.722867 17144 tablet_bootstrap.cc:492] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:34.723367 17144 ts_tablet_manager.cc:1397] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.138s user 0.008s sys 0.004s
I20260923 17:31:34.738535 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]) successfully deleted
I20260923 17:31:34.770215 17143 tablet_bootstrap.cc:492] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:34.771162 17143 ts_tablet_manager.cc:1397] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.336s user 0.009s sys 0.009s
I20260923 17:31:34.773854 17143 raft_consensus.cc:348] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:34.774703 17143 raft_consensus.cc:374] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.775003 17143 raft_consensus.cc:729] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.775661 17143 consensus_queue.cc:260] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:34.778227 17143 ts_tablet_manager.cc:1428] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260923 17:31:34.779150 17146 tablet_bootstrap.cc:492] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:34.779225 17143 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:34.779562 17146 ts_tablet_manager.cc:1397] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.181s user 0.005s sys 0.006s
I20260923 17:31:34.781638 17146 raft_consensus.cc:348] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.782266 17146 raft_consensus.cc:374] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.782503 17146 raft_consensus.cc:729] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.783095 17146 consensus_queue.cc:260] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.785329 17144 raft_consensus.cc:348] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.786047 17144 raft_consensus.cc:374] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:34.799022 16385 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 250) took 2319 ms. Trace:
I20260923 17:31:34.801577 16388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 248) took 2591 ms. Trace:
I20260923 17:31:34.802695 16380 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 251) took 2253 ms. Trace:
I20260923 17:31:34.801913 16388 rpcz_store.cc:276] 0923 17:31:32.210105 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.370193 (+2160088us) service_pool.cc:227] Handling call
0923 17:31:34.416135 (+ 45942us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:34.450786 (+ 34651us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:34.536707 (+ 85921us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.536733 (+ 26us) write_op.cc:454] Released schema lock
0923 17:31:34.536920 (+ 187us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:34.768071 (+231151us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:34.801524 (+ 33453us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.508551 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.508943 (+ 392us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.508970 (+ 27us) write_op.cc:435] Acquired schema lock
0923 17:31:34.508984 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:34.509287 (+ 303us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.509357 (+ 70us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.509382 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.509576 (+ 194us) tablet.cc:699] Row locks acquired
0923 17:31:34.509596 (+ 20us) write_op.cc:260] PREPARE: finished
0923 17:31:34.509713 (+ 117us) write_op.cc:270] Start()
0923 17:31:34.509794 (+ 81us) write_op.cc:276] Timestamp: P: 1790184694509677 usec, L: 0
0923 17:31:34.509811 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.510313 (+ 502us) log.cc:842] Serialized 433 byte log entry
0923 17:31:34.528762 (+ 18449us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.529034 (+ 272us) write_op.cc:301] APPLY: starting
0923 17:31:34.529086 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.529188 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.529274 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.529660 (+ 386us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.529791 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:34.529830 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:34.529996 (+ 166us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.530132 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.530204 (+ 72us) write_op.cc:454] Released schema lock
0923 17:31:34.530233 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":14589,"child_traces":[["op",{"apply.queue_time_us":149,"apply.run_cpu_time_us":1378,"apply.run_wall_time_us":1377,"num_ops":2,"prepare.queue_time_us":56085,"prepare.run_cpu_time_us":2091,"prepare.run_wall_time_us":2091,"raft.queue_time_us":3124,"raft.run_cpu_time_us":1122,"raft.run_wall_time_us":1122,"replication_time_us":18874}]]}
I20260923 17:31:34.813628 17146 ts_tablet_manager.cc:1428] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260923 17:31:34.839553 18255 raft_consensus.cc:482] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.840103 18255 raft_consensus.cc:504] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.842041 18255 leader_election.cc:288] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:34.844367 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77ef164608f4ced8b1c2ecab6f88c1a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:34.846766 16972 leader_election.cc:341] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:34.812659 16375 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 354f66c852b94b1d9f44b6bc012cf81b consensus state has no leader
I20260923 17:31:34.857053 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:29 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.857381 16388 master_service.cc:946] ReplaceTablet: received request to replace tablet 191292e60d0d46fb89b52390feb48f80 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:34.845002 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:34.867585 16386 catalog_manager.cc:5495] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 reported cstate change: term changed from 1 to 2, leader changed from be93fda0843b44f5abc49e6f35c69217 (127.15.246.129) to c36e79694e154f84a46980719217d7f2 (127.15.246.130). New cstate: current_term: 2 leader_uuid: "c36e79694e154f84a46980719217d7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:34.869107 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:34.871472 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:34.803097 16380 rpcz_store.cc:276] 0923 17:31:32.549803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.487746 (+1937943us) service_pool.cc:227] Handling call
0923 17:31:34.714181 (+226435us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:34.715812 (+ 1631us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:34.774485 (+ 58673us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.774513 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:34.774717 (+ 204us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:34.775015 (+ 298us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:34.802643 (+ 27628us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.717019 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.717465 (+ 446us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.717501 (+ 36us) write_op.cc:435] Acquired schema lock
0923 17:31:34.717517 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:34.718040 (+ 523us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.718086 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.718145 (+ 59us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.718508 (+ 363us) tablet.cc:699] Row locks acquired
0923 17:31:34.718564 (+ 56us) write_op.cc:260] PREPARE: finished
0923 17:31:34.718808 (+ 244us) write_op.cc:270] Start()
0923 17:31:34.718947 (+ 139us) write_op.cc:276] Timestamp: P: 1790184694718730 usec, L: 0
0923 17:31:34.718995 (+ 48us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.719558 (+ 563us) log.cc:842] Serialized 433 byte log entry
0923 17:31:34.726713 (+ 7155us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.726971 (+ 258us) write_op.cc:301] APPLY: starting
0923 17:31:34.727022 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.727119 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.727221 (+ 102us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.727585 (+ 364us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.727710 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:34.727755 (+ 45us) write_op.cc:312] APPLY: finished
0923 17:31:34.727908 (+ 153us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.728034 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.728097 (+ 63us) write_op.cc:454] Released schema lock
0923 17:31:34.728129 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":7582,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":1337,"apply.run_wall_time_us":1336,"num_ops":2,"prepare.queue_time_us":79,"prepare.run_cpu_time_us":2843,"prepare.run_wall_time_us":5197,"raft.queue_time_us":74,"raft.run_cpu_time_us":563,"raft.run_wall_time_us":563,"replication_time_us":7620}]]}
I20260923 17:31:34.872711 18255 raft_consensus.cc:482] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:34.873245 18255 raft_consensus.cc:504] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.873931 16380 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48560:
name: "default.Table_d33676ffb7aa46cea6ec0d2cff628522"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:31:34.875389 16380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d33676ffb7aa46cea6ec0d2cff628522 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:31:34.799453 16385 rpcz_store.cc:276] 0923 17:31:32.479480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.474193 (+1994713us) service_pool.cc:227] Handling call
0923 17:31:34.695395 (+221202us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:34.697054 (+ 1659us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:34.713459 (+ 16405us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.713483 (+ 24us) write_op.cc:454] Released schema lock
0923 17:31:34.713739 (+ 256us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:34.714019 (+ 280us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:34.798962 (+ 84943us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.698277 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.698679 (+ 402us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.698705 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:34.698720 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:34.699011 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.699055 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.699079 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.699279 (+ 200us) tablet.cc:699] Row locks acquired
0923 17:31:34.699295 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:34.699419 (+ 124us) write_op.cc:270] Start()
0923 17:31:34.699501 (+ 82us) write_op.cc:276] Timestamp: P: 1790184694699372 usec, L: 0
0923 17:31:34.699519 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.699993 (+ 474us) log.cc:842] Serialized 433 byte log entry
0923 17:31:34.708392 (+ 8399us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.708676 (+ 284us) write_op.cc:301] APPLY: starting
0923 17:31:34.708730 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.708847 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.708947 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.709463 (+ 516us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.709584 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:34.709620 (+ 36us) write_op.cc:312] APPLY: finished
0923 17:31:34.709863 (+ 243us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.710059 (+ 196us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.710133 (+ 74us) write_op.cc:454] Released schema lock
0923 17:31:34.710163 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":150,"apply.run_cpu_time_us":1654,"apply.run_wall_time_us":1654,"num_ops":2,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":1983,"prepare.run_wall_time_us":1982,"raft.queue_time_us":1155,"raft.run_cpu_time_us":613,"raft.run_wall_time_us":612,"replication_time_us":8791}]]}
I20260923 17:31:34.876873 16385 master_service.cc:946] ReplaceTablet: received request to replace tablet 6e20fc94eecf4a14840ca88177070e33 from {username='slave'} at 127.0.0.1:48560
I20260923 17:31:34.877192 18255 leader_election.cc:288] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:34.879221 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d4abafcef943c08ae0755bae06f32c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:34.880640 16972 leader_election.cc:341] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:34.880726 16381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 38) took 2260 ms (client timeout 1000 ms). Trace:
W20260923 17:31:34.881057 16381 rpcz_store.cc:269] 0923 17:31:32.619929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.777797 (+157868us) service_pool.cc:227] Handling call
0923 17:31:34.880664 (+2102867us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":58}
I20260923 17:31:34.882233 16381 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:40626 (request call id 1) took 2649 ms. Trace:
I20260923 17:31:34.882514 16381 rpcz_store.cc:276] 0923 17:31:32.232523 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.881700 (+2649177us) service_pool.cc:227] Handling call
0923 17:31:34.882166 (+ 466us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:34.888021 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:31 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.910100 16386 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:34.931568 18222 raft_consensus.cc:2229] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:34.932363 18222 raft_consensus.cc:2258] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:34.934562 18222 ts_tablet_manager.cc:1905] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:34.946730 16380 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48560 (request call id 255) took 1965 ms. Trace:
I20260923 17:31:34.947065 16380 rpcz_store.cc:276] 0923 17:31:32.981287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.873297 (+1892010us) service_pool.cc:227] Handling call
0923 17:31:34.875797 (+ 2500us) catalog_manager.cc:2075] Acquired catalog manager lock
0923 17:31:34.889645 (+ 13848us) catalog_manager.cc:2121] Created new table and tablet info
0923 17:31:34.946150 (+ 56505us) catalog_manager.cc:2181] Wrote table and tablets to system table
0923 17:31:34.946426 (+ 276us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0923 17:31:34.946673 (+ 247us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.892439 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.892858 (+ 419us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.892884 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:34.892895 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:34.893173 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.893216 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.893240 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.893440 (+ 200us) tablet.cc:699] Row locks acquired
0923 17:31:34.893452 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:34.893563 (+ 111us) write_op.cc:270] Start()
0923 17:31:34.893649 (+ 86us) write_op.cc:276] Timestamp: P: 1790184694893504 usec, L: 0
0923 17:31:34.893664 (+ 15us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.894129 (+ 465us) log.cc:842] Serialized 433 byte log entry
0923 17:31:34.944024 (+ 49895us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.944361 (+ 337us) write_op.cc:301] APPLY: starting
0923 17:31:34.944424 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.944526 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.944624 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.945143 (+ 519us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.945282 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0923 17:31:34.945320 (+ 38us) write_op.cc:312] APPLY: finished
0923 17:31:34.945508 (+ 188us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.945684 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.945754 (+ 70us) write_op.cc:454] Released schema lock
0923 17:31:34.945791 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:34.945971 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.945993 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":1804,"apply.run_wall_time_us":1803,"num_ops":2,"prepare.queue_time_us":1718,"prepare.run_cpu_time_us":2032,"prepare.run_wall_time_us":4595,"raft.queue_time_us":130,"raft.run_cpu_time_us":1098,"raft.run_wall_time_us":1100,"replication_time_us":50268}]]}
I20260923 17:31:34.951201 16385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 6e20fc94eecf4a14840ca88177070e33
I20260923 17:31:34.952448 16385 catalog_manager.cc:6565] ReplaceTablet: tablet 6e20fc94eecf4a14840ca88177070e33 deleted and replaced by tablet 02f70ce7043e46c396e9a0c9e26b1615
I20260923 17:31:34.952741 16385 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 256) took 1520 ms. Trace:
I20260923 17:31:34.952998 16385 rpcz_store.cc:276] 0923 17:31:33.431913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.876542 (+1444629us) service_pool.cc:227] Handling call
0923 17:31:34.952691 (+ 76149us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.903564 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.904013 (+ 449us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.904038 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:34.904054 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:34.904429 (+ 375us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.904470 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.904494 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.904697 (+ 203us) tablet.cc:699] Row locks acquired
0923 17:31:34.904788 (+ 91us) write_op.cc:260] PREPARE: finished
0923 17:31:34.904912 (+ 124us) write_op.cc:270] Start()
0923 17:31:34.904998 (+ 86us) write_op.cc:276] Timestamp: P: 1790184694904851 usec, L: 0
0923 17:31:34.905017 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.905581 (+ 564us) log.cc:842] Serialized 631 byte log entry
0923 17:31:34.943803 (+ 38222us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b5400012320. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594af0 00007f912759f028 00007f912759ec6a 00007f912760acd5 00007f912760b179 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1
0923 17:31:34.947448 (+ 3645us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.948119 (+ 671us) write_op.cc:301] APPLY: starting
0923 17:31:34.948172 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.948311 (+ 139us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.948327 (+ 16us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.949979 (+ 1652us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.950201 (+ 222us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:34.950247 (+ 46us) write_op.cc:312] APPLY: finished
0923 17:31:34.950428 (+ 181us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.950568 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.950654 (+ 86us) write_op.cc:454] Released schema lock
0923 17:31:34.950684 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:34.950861 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.950886 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":526,"apply.run_cpu_time_us":1712,"apply.run_wall_time_us":2943,"num_ops":2,"prepare.queue_time_us":5763,"prepare.run_cpu_time_us":3418,"prepare.run_wall_time_us":48228,"raft.queue_time_us":71,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"replication_time_us":42362,"spinlock_wait_cycles":82484608}]]}
I20260923 17:31:34.959385 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e20fc94eecf4a14840ca88177070e33 with delete_type TABLET_DATA_DELETED (replaced by tablet 02f70ce7043e46c396e9a0c9e26b1615) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:34.961637 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e20fc94eecf4a14840ca88177070e33 with delete_type TABLET_DATA_DELETED (replaced by tablet 02f70ce7043e46c396e9a0c9e26b1615) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:34.962522 16346 master-stress-test.cc:580] Killed test thread 1/10
I20260923 17:31:34.963008 16346 master-stress-test.cc:578] Killing test thread 2/10
I20260923 17:31:34.963315 16346 master-stress-test.cc:580] Killed test thread 2/10
I20260923 17:31:34.963515 16346 master-stress-test.cc:578] Killing test thread 3/10
I20260923 17:31:34.963762 16346 master-stress-test.cc:580] Killed test thread 3/10
I20260923 17:31:34.963969 16346 master-stress-test.cc:578] Killing test thread 4/10
I20260923 17:31:34.964202 16346 master-stress-test.cc:580] Killed test thread 4/10
I20260923 17:31:34.964476 16346 master-stress-test.cc:578] Killing test thread 5/10
I20260923 17:31:34.786302 17144 raft_consensus.cc:729] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:34.969816 17144 consensus_queue.cc:260] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:34.974180 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e20fc94eecf4a14840ca88177070e33 with delete_type TABLET_DATA_DELETED (replaced by tablet 02f70ce7043e46c396e9a0c9e26b1615) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:34.977743 16891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d4abafcef943c08ae0755bae06f32c" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:35.007791 16972 leader_election.cc:341] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:35.008513 16972 leader_election.cc:302] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:35.009282 18255 raft_consensus.cc:2726] T c3d4abafcef943c08ae0755bae06f32c P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:35.015920 16678 leader_election.cc:302] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c; no voters:
I20260923 17:31:35.019634 18245 raft_consensus.cc:2781] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:35.025166 18245 raft_consensus.cc:686] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Becoming Leader. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Running, Role: LEADER
I20260923 17:31:35.026142 18245 consensus_queue.cc:237] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.017670 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77ef164608f4ced8b1c2ecab6f88c1a" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:35.040652 16972 leader_election.cc:341] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:35.041198 16972 leader_election.cc:302] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:35.041927 18255 raft_consensus.cc:2726] T b77ef164608f4ced8b1c2ecab6f88c1a P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:35.055912 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a2f1143dcd64b2dbd25e17787096903" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:35.057659 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef890c51e92244f4b97c490f24250679" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:35.048509 18221 ts_tablet_manager.cc:1918] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:35.078375 18221 log.cc:1196] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:35.085776 16743 raft_consensus.cc:1261] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Refusing update from remote peer c36e79694e154f84a46980719217d7f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260923 17:31:35.086112 16743 pending_rounds.cc:76] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Aborting all ops after (but not including) 0
I20260923 17:31:35.086294 16743 pending_rounds.cc:97] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260923 17:31:35.086517 16743 raft_consensus.cc:2866] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260923 17:31:35.090905 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254793351e5245af9a4fa1642cad5e82" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
W20260923 17:31:35.092005 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 74948us
I20260923 17:31:35.094233 18221 ts_tablet_manager.cc:1939] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:35.095254 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 236) took 3571 ms. Trace:
I20260923 17:31:35.095757 18221 rpcz_store.cc:276] 0923 17:31:31.523752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.524297 (+ 545us) service_pool.cc:227] Handling call
0923 17:31:33.791519 (+2267222us) ts_tablet_manager.cc:1145] Deleting tablet f05a6f83314d433a9502e0c00ba8ad09
0923 17:31:33.791566 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:34.429815 (+638249us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.048204 (+618389us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.095188 (+ 46984us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63214,"tablet-delete.queue_time_us":2258267}
I20260923 17:31:35.097551 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet f05a6f83314d433a9502e0c00ba8ad09 (table default.Table_e8bd5a98bd7a4646a36e054bf77ec8e2 [id=c1d8ac06c0024b1fae1756dd3c72ed70]) successfully deleted
I20260923 17:31:35.101917 18222 ts_tablet_manager.cc:1918] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:35.102675 18222 log.cc:1196] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/456273cf57314cb8a05c8e765543913f
I20260923 17:31:35.103518 18222 ts_tablet_manager.cc:1939] T 456273cf57314cb8a05c8e765543913f P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:35.126784 16886 raft_consensus.cc:3037] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:35.142161 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 281) took 2853 ms. Trace:
I20260923 17:31:35.142652 18222 rpcz_store.cc:276] 0923 17:31:32.288639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.288743 (+ 104us) service_pool.cc:227] Handling call
0923 17:31:34.584038 (+2295295us) ts_tablet_manager.cc:1145] Deleting tablet 456273cf57314cb8a05c8e765543913f
0923 17:31:34.584084 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:34.987350 (+403266us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.101643 (+114293us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.142095 (+ 40452us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":328784,"tablet-delete.queue_time_us":2294705}
I20260923 17:31:35.144280 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]) successfully deleted
I20260923 17:31:35.145560 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 284) took 2770 ms. Trace:
I20260923 17:31:35.158897 18219 tablet_replica.cc:330] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:35.159751 17146 tablet_bootstrap.cc:492] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:35.159912 18219 raft_consensus.cc:2229] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:35.160689 18219 raft_consensus.cc:2258] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:35.162597 18219 ts_tablet_manager.cc:1905] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:35.164836 17146 tablet_bootstrap.cc:654] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:35.170217 16826 leader_election.cc:302] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:35.171504 18246 raft_consensus.cc:2726] T 8a2f1143dcd64b2dbd25e17787096903 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:35.172744 16826 leader_election.cc:302] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:35.173768 18246 raft_consensus.cc:2726] T ef890c51e92244f4b97c490f24250679 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:35.164142 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 63410.7us
I20260923 17:31:35.186813 17146 tablet_bootstrap.cc:492] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:35.187271 17146 ts_tablet_manager.cc:1397] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.000s
I20260923 17:31:35.189801 17146 raft_consensus.cc:348] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.190507 17146 raft_consensus.cc:374] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:35.190752 17146 raft_consensus.cc:729] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:35.191349 17146 consensus_queue.cc:260] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.174383 17144 ts_tablet_manager.cc:1428] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.451s user 0.008s sys 0.000s
I20260923 17:31:35.194700 17144 tablet_bootstrap.cc:492] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
W20260923 17:31:35.197751 16826 leader_election.cc:341] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: f05a6f83314d433a9502e0c00ba8ad09
I20260923 17:31:35.198175 16826 leader_election.cc:302] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
W20260923 17:31:35.198712 16826 raft_consensus.cc:2670] T f05a6f83314d433a9502e0c00ba8ad09 P c36e79694e154f84a46980719217d7f2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260923 17:31:35.201725 18246 consensus_queue.cc:1035] T a84e322a4de24cbabf450fceb098d6e7 P c36e79694e154f84a46980719217d7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:35.205385 16826 leader_election.cc:341] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Not found: Tablet not found: c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:35.081664 17143 tablet_bootstrap.cc:654] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:35.205876 16826 leader_election.cc:302] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:35.206876 18246 raft_consensus.cc:2726] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:35.053757 18343 raft_consensus.cc:482] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.184408 18222 rpcz_store.cc:276] 0923 17:31:32.375238 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.375412 (+ 174us) service_pool.cc:227] Handling call
0923 17:31:35.145155 (+2769743us) ts_tablet_manager.cc:1145] Deleting tablet 1feebaa76bcd4ff5b707814191cb3341
0923 17:31:35.145198 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.145502 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2769075}
W20260923 17:31:35.213932 16372 catalog_manager.cc:4821] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 1feebaa76bcd4ff5b707814191cb3341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1feebaa76bcd4ff5b707814191cb3341 already in progress: creating tablet
I20260923 17:31:35.216789 18222 tablet_replica.cc:330] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
I20260923 17:31:35.216562 17146 ts_tablet_manager.cc:1428] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.029s user 0.008s sys 0.000s
I20260923 17:31:35.225330 18255 raft_consensus.cc:482] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.225818 18255 raft_consensus.cc:504] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.227721 18255 leader_election.cc:288] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:35.230058 16826 leader_election.cc:302] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:35.229668 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6398fa526d9c432e8c48ebae0b4612aa" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:35.232206 18222 raft_consensus.cc:2229] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
W20260923 17:31:35.236722 16972 leader_election.cc:341] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:35.272425 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6398fa526d9c432e8c48ebae0b4612aa" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:35.276325 16388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet 191292e60d0d46fb89b52390feb48f80
I20260923 17:31:35.277735 16388 catalog_manager.cc:6565] ReplaceTablet: tablet 191292e60d0d46fb89b52390feb48f80 deleted and replaced by tablet 18a117533c474766ad01c9a7bc3ef422
I20260923 17:31:35.278115 16388 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48560 (request call id 254) took 2336 ms. Trace:
I20260923 17:31:35.278563 17012 tablet_service.cc:1514] Processing DeleteTablet for tablet 191292e60d0d46fb89b52390feb48f80 with delete_type TABLET_DATA_DELETED (replaced by tablet 18a117533c474766ad01c9a7bc3ef422) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:35.278385 16388 rpcz_store.cc:276] 0923 17:31:32.942114 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.857138 (+1915024us) service_pool.cc:227] Handling call
0923 17:31:35.278059 (+420921us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:35.105855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:35.106301 (+ 446us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:35.106331 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:35.106346 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:35.106684 (+ 338us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:35.106730 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:35.106753 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:35.106960 (+ 207us) tablet.cc:699] Row locks acquired
0923 17:31:35.106972 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:35.107120 (+ 148us) write_op.cc:270] Start()
0923 17:31:35.107211 (+ 91us) write_op.cc:276] Timestamp: P: 1790184695107078 usec, L: 0
0923 17:31:35.107228 (+ 17us) op_driver.cc:331] REPLICATION: starting
0923 17:31:35.256325 (+149097us) log.cc:842] Serialized 665 byte log entry
0923 17:31:35.258422 (+ 2097us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000118d4. stack: 00007f9120688033 00007f9120687f30 00007f9120209534 00000000004db2b4 00000000004db249 00007f9127594471 00007f912760b198 00007f9127a6cade 00007f9127a6c2b8 00007f9127a70ba1 00007f9127a70b59 00007f9127a70ae9 00007f9127a70ab1 00007f9127a6fdad 0000000000580784 00000000005805b9
0923 17:31:35.267462 (+ 9040us) op_driver.cc:444] REPLICATION: finished
0923 17:31:35.267850 (+ 388us) write_op.cc:301] APPLY: starting
0923 17:31:35.267903 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:35.268005 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:35.268024 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:35.268373 (+ 349us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:35.268594 (+ 221us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:35.268634 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:35.268814 (+ 180us) log.cc:842] Serialized 29 byte log entry
0923 17:31:35.269504 (+ 690us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.269572 (+ 68us) write_op.cc:454] Released schema lock
0923 17:31:35.269603 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:35.270250 (+ 647us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.270275 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6101,"child_traces":[["op",{"apply.queue_time_us":229,"apply.run_cpu_time_us":1634,"apply.run_wall_time_us":2595,"num_ops":2,"prepare.queue_time_us":71413,"raft.queue_time_us":100,"raft.run_cpu_time_us":981,"raft.run_wall_time_us":1454,"replication_time_us":160156,"spinlock_wait_cycles":341122944}]]}
W20260923 17:31:35.287276 16972 leader_election.cc:341] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:35.287883 16972 leader_election.cc:302] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:35.289055 18255 raft_consensus.cc:2726] T 6398fa526d9c432e8c48ebae0b4612aa P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:35.281263 18343 raft_consensus.cc:504] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:35.293949 16892 raft_consensus.cc:3037] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:35.298190 18222 raft_consensus.cc:2258] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:35.300789 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 191292e60d0d46fb89b52390feb48f80 with delete_type TABLET_DATA_DELETED (replaced by tablet 18a117533c474766ad01c9a7bc3ef422) from {username='slave'} at 127.0.0.1:42676
I20260923 17:31:35.315496 17023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:35.327543 18222 ts_tablet_manager.cc:1905] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:35.308723 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:35.334141 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:35.335552 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e]). Expected version 1 got 0
I20260923 17:31:35.359457 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:35.360208 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:35.373123 16383 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 070d38df3f0f43edab3826835c9503ce (table default.Table_04688becafca4883b1c56cbe6ee8132d [id=ef4aea9a69ae43e48cf913c554fcdac9]). Expected version 1 got 0
W20260923 17:31:35.380780 16826 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 150798us
I20260923 17:31:35.408704 18219 ts_tablet_manager.cc:1918] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260923 17:31:35.345366 16677 consensus_peers.cc:487] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 -> Peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Couldn't send request to peer c36e79694e154f84a46980719217d7f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:31:35.369078 18313 raft_consensus.cc:482] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.429121 18313 raft_consensus.cc:504] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.431049 18313 leader_election.cc:288] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
W20260923 17:31:35.431756 16677 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.130:35433, user_credentials={real_user=slave}} blocked reactor thread for 123972us
W20260923 17:31:35.433167 16677 leader_election.cc:341] T 6e20fc94eecf4a14840ca88177070e33 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:35.386152 18367 tablet.cc:1722] T 7dbd8eb0ab9243bd81c3d6a893f18aea P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260923 17:31:35.378144 16678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.246.131:42559, user_credentials={real_user=slave}} blocked reactor thread for 52641.4us
I20260923 17:31:35.411746 18343 leader_election.cc:288] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:35.438599 18343 raft_consensus.cc:482] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.439126 18343 raft_consensus.cc:504] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:35.443213 18190 consensus_queue.cc:1035] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:31:35.448964 18343 leader_election.cc:288] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:35.454438 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 235) took 3931 ms. Trace:
I20260923 17:31:35.454950 18221 rpcz_store.cc:276] 0923 17:31:31.522857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.523342 (+ 485us) service_pool.cc:227] Handling call
0923 17:31:35.379064 (+3855722us) ts_tablet_manager.cc:1145] Deleting tablet f1ccdf1bd7a64f51965e0a6dc7e033c9
0923 17:31:35.379791 (+ 727us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.454353 (+ 74562us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5,"tablet-delete.queue_time_us":3842437}
I20260923 17:31:35.456532 18221 tablet_replica.cc:330] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:35.457746 18221 raft_consensus.cc:2229] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:35.389920 17144 tablet_bootstrap.cc:654] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:35.513737 18313 consensus_queue.cc:1035] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260923 17:31:35.517453 16678 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 7c115e29673747f8b0712d441ac993fb
I20260923 17:31:35.531701 18132 mvcc.cc:205] Tried to move back new op lower bound from 7332596479067541504 to 7332596397761437696. Current Snapshot: MvccSnapshot[applied={T|T < 7332596479067541504 or (T in {7332596479067541504})}]
I20260923 17:31:35.447952 17027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba0ed72a55d74ab5b8b22fba90848094" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:35.467407 17146 tablet_bootstrap.cc:492] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:35.533362 18219 log.cc:1196] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/1feebaa76bcd4ff5b707814191cb3341
I20260923 17:31:35.540851 17146 tablet_bootstrap.cc:654] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:35.556903 18219 ts_tablet_manager.cc:1939] T 1feebaa76bcd4ff5b707814191cb3341 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:35.559219 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 239) took 3184 ms. Trace:
I20260923 17:31:35.571336 18245 consensus_queue.cc:1035] T 4cd483bcad624c9a86503bdf15b9d6f8 P be93fda0843b44f5abc49e6f35c69217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:31:35.580971 17144 tablet_bootstrap.cc:492] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:35.581547 17144 ts_tablet_manager.cc:1397] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.387s user 0.007s sys 0.009s
I20260923 17:31:35.585471 17144 raft_consensus.cc:348] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.606868 17012 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7eaad1c706470da47819c5e373b7f4 (DEFAULT_TABLE table=default.Table_5e18039362c44fc1898d2a728f318340 [id=88dda568da714669afcfe5527b387c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.613271 17010 tablet_service.cc:1467] Processing CreateTablet for tablet ef7e043d95ec44dbb982dd5d941b79b3 (DEFAULT_TABLE table=default.Table_b0f775b6c72d4f55bfeba14c7a8c00e2 [id=45e461c3de8c4d959dadf77c627e9e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.634536 18219 rpcz_store.cc:276] 0923 17:31:32.374452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.395437 (+ 20985us) service_pool.cc:227] Handling call
0923 17:31:34.675476 (+2280039us) ts_tablet_manager.cc:1145] Deleting tablet 1feebaa76bcd4ff5b707814191cb3341
0923 17:31:35.158735 (+483259us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.170747 (+ 12012us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.408413 (+237666us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.559150 (+150737us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2267003}
I20260923 17:31:35.636524 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 1feebaa76bcd4ff5b707814191cb3341 (table default.Table_625e432250494eb799feeaad03f7112f [id=458611a182ba4d13824f5347aacf41c9]) successfully deleted
I20260923 17:31:35.657589 17015 tablet_service.cc:1467] Processing CreateTablet for tablet 473c1ab39b6542f989b35b72bea3b8b2 (DEFAULT_TABLE table=default.Table_3f2ddad73bd74b15a8b9831399e83554 [id=c541d50b24954f6eadc83cda70a43492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.676313 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:35.678028 16379 catalog_manager.cc:5495] T cfa712f9362e416e9cc6defcd8198bcd P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } }
I20260923 17:31:35.685280 17012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7eaad1c706470da47819c5e373b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.688755 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7e043d95ec44dbb982dd5d941b79b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.710358 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:35.714072 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:35.715581 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473c1ab39b6542f989b35b72bea3b8b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.737638 16379 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:35.753839 18219 tablet_replica.cc:330] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:35.755100 18219 raft_consensus.cc:2229] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:35.604303 16712 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7eaad1c706470da47819c5e373b7f4 (DEFAULT_TABLE table=default.Table_5e18039362c44fc1898d2a728f318340 [id=88dda568da714669afcfe5527b387c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.766757 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7eaad1c706470da47819c5e373b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.610828 16714 tablet_service.cc:1467] Processing CreateTablet for tablet ef7e043d95ec44dbb982dd5d941b79b3 (DEFAULT_TABLE table=default.Table_b0f775b6c72d4f55bfeba14c7a8c00e2 [id=45e461c3de8c4d959dadf77c627e9e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.768826 16714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7e043d95ec44dbb982dd5d941b79b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.617136 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 473c1ab39b6542f989b35b72bea3b8b2 (DEFAULT_TABLE table=default.Table_3f2ddad73bd74b15a8b9831399e83554 [id=c541d50b24954f6eadc83cda70a43492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.772961 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473c1ab39b6542f989b35b72bea3b8b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.637634 18210 raft_consensus.cc:482] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.780608 18210 raft_consensus.cc:504] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.782673 18210 leader_election.cc:288] T 1feebaa76bcd4ff5b707814191cb3341 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:35.681989 18243 raft_consensus.cc:482] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.800729 18243 raft_consensus.cc:504] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.688555 18222 ts_tablet_manager.cc:1918] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:35.762979 16739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d71b22338241839b184e50d7fdd8f6" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:35.808782 18222 log.cc:1196] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/db3504813e9c4698a546b6d6fa0fbcb6
I20260923 17:31:35.682420 18313 raft_consensus.cc:482] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.809612 18222 ts_tablet_manager.cc:1939] T db3504813e9c4698a546b6d6fa0fbcb6 P be93fda0843b44f5abc49e6f35c69217: Deleting consensus metadata
I20260923 17:31:35.809749 18313 raft_consensus.cc:504] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.812572 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 293) took 1440 ms. Trace:
I20260923 17:31:35.822942 18222 rpcz_store.cc:276] 0923 17:31:34.372458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.372563 (+ 105us) service_pool.cc:227] Handling call
0923 17:31:35.216519 (+843956us) ts_tablet_manager.cc:1145] Deleting tablet db3504813e9c4698a546b6d6fa0fbcb6
0923 17:31:35.216583 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.556828 (+340245us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.688276 (+131448us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:35.812499 (+124223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":843301}
I20260923 17:31:35.824432 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 296) took 1396 ms. Trace:
I20260923 17:31:35.824553 16372 catalog_manager.cc:4841] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]) successfully deleted
I20260923 17:31:35.824731 18222 rpcz_store.cc:276] 0923 17:31:34.427988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.428131 (+ 143us) service_pool.cc:227] Handling call
0923 17:31:35.824102 (+1395971us) ts_tablet_manager.cc:1145] Deleting tablet 051f2e478a324df98647d90107d55533
0923 17:31:35.824149 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.824366 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1395285}
I20260923 17:31:35.825829 18222 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42676 (request call id 295) took 1402 ms. Trace:
W20260923 17:31:35.825986 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 051f2e478a324df98647d90107d55533 because the tablet was not found. No further retry: Not found: Tablet not found: 051f2e478a324df98647d90107d55533
I20260923 17:31:35.826129 18222 rpcz_store.cc:276] 0923 17:31:34.423254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.427655 (+ 4401us) service_pool.cc:227] Handling call
0923 17:31:35.825577 (+1397922us) ts_tablet_manager.cc:1145] Deleting tablet 5c929b8beaaf4c7aa668852b63af2204
0923 17:31:35.825617 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:35.825779 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1383755}
I20260923 17:31:35.827104 18222 tablet_replica.cc:330] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: stopping tablet replica
W20260923 17:31:35.827519 16372 catalog_manager.cc:4808] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): delete failed for tablet 5c929b8beaaf4c7aa668852b63af2204 because the tablet was not found. No further retry: Not found: Tablet not found: 5c929b8beaaf4c7aa668852b63af2204
I20260923 17:31:35.811977 18313 leader_election.cc:288] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:35.827988 18222 raft_consensus.cc:2229] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:35.829828 18222 raft_consensus.cc:2258] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:35.831920 18222 ts_tablet_manager.cc:1905] T 006895667f324d049debbebb132ed09b P be93fda0843b44f5abc49e6f35c69217: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:35.876605 16372 catalog_manager.cc:4808] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet f1ccdf1bd7a64f51965e0a6dc7e033c9 because the tablet was not found. No further retry: Not found: Tablet not found: f1ccdf1bd7a64f51965e0a6dc7e033c9
I20260923 17:31:35.762658 17146 tablet_bootstrap.cc:492] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:35.775887 18255 raft_consensus.cc:482] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:35.845113 17011 tablet_service.cc:1467] Processing CreateTablet for tablet 02f70ce7043e46c396e9a0c9e26b1615 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.862226 17008 tablet_service.cc:1467] Processing CreateTablet for tablet 18a117533c474766ad01c9a7bc3ef422 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.933007 17146 ts_tablet_manager.cc:1397] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.466s user 0.029s sys 0.012s
I20260923 17:31:35.866391 17021 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8d088d9f2430e9c51f8ff62a860bd (DEFAULT_TABLE table=default.Table_d33676ffb7aa46cea6ec0d2cff628522 [id=7d0ed71a404046af8f9f5b6e80e27b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.933003 18255 raft_consensus.cc:504] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.935217 18255 leader_election.cc:288] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:35.935350 17146 raft_consensus.cc:348] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.936029 17146 raft_consensus.cc:374] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:35.936272 17146 raft_consensus.cc:729] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:35.936779 17146 consensus_queue.cc:260] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.938874 17146 ts_tablet_manager.cc:1428] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260923 17:31:35.821221 18243 leader_election.cc:288] T 80b11a6fd84840178b81d2b7610c7351 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:35.836895 17144 raft_consensus.cc:374] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:35.841816 16726 tablet_service.cc:1467] Processing CreateTablet for tablet 02f70ce7043e46c396e9a0c9e26b1615 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.945376 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f70ce7043e46c396e9a0c9e26b1615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.843472 16713 tablet_service.cc:1467] Processing CreateTablet for tablet 18a117533c474766ad01c9a7bc3ef422 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.948992 16713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a117533c474766ad01c9a7bc3ef422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.852427 18381 tablet.cc:1722] T 6a90f7348a964a7f871e93dde7ab82f2 P be93fda0843b44f5abc49e6f35c69217: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:35.864771 16717 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8d088d9f2430e9c51f8ff62a860bd (DEFAULT_TABLE table=default.Table_d33676ffb7aa46cea6ec0d2cff628522 [id=7d0ed71a404046af8f9f5b6e80e27b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.961288 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8d088d9f2430e9c51f8ff62a860bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.893174 18351 raft_consensus.cc:482] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c36e79694e154f84a46980719217d7f2)
I20260923 17:31:35.964684 18351 raft_consensus.cc:504] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:35.968446 18351 leader_election.cc:288] T a84e322a4de24cbabf450fceb098d6e7 P be93fda0843b44f5abc49e6f35c69217 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:35.992374 17144 raft_consensus.cc:729] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:35.993325 17144 consensus_queue.cc:260] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:35.932960 18219 raft_consensus.cc:2258] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:36.004823 16743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be93fda0843b44f5abc49e6f35c69217" is_pre_election: true
I20260923 17:31:36.030232 16743 raft_consensus.cc:2454] T 354f66c852b94b1d9f44b6bc012cf81b P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5c071cdf65042a59cad226a15a5532c in term 0.
I20260923 17:31:35.901845 16866 tablet_service.cc:1467] Processing CreateTablet for tablet ef7e043d95ec44dbb982dd5d941b79b3 (DEFAULT_TABLE table=default.Table_b0f775b6c72d4f55bfeba14c7a8c00e2 [id=45e461c3de8c4d959dadf77c627e9e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:36.045689 16866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7e043d95ec44dbb982dd5d941b79b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.903060 16862 tablet_service.cc:1467] Processing CreateTablet for tablet 18a117533c474766ad01c9a7bc3ef422 (DEFAULT_TABLE table=default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:36.047389 16862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a117533c474766ad01c9a7bc3ef422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:35.903538 16869 tablet_service.cc:1467] Processing CreateTablet for tablet 473c1ab39b6542f989b35b72bea3b8b2 (DEFAULT_TABLE table=default.Table_3f2ddad73bd74b15a8b9831399e83554 [id=c541d50b24954f6eadc83cda70a43492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.901845 16865 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7eaad1c706470da47819c5e373b7f4 (DEFAULT_TABLE table=default.Table_5e18039362c44fc1898d2a728f318340 [id=88dda568da714669afcfe5527b387c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.901892 16861 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8d088d9f2430e9c51f8ff62a860bd (DEFAULT_TABLE table=default.Table_d33676ffb7aa46cea6ec0d2cff628522 [id=7d0ed71a404046af8f9f5b6e80e27b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.904623 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 02f70ce7043e46c396e9a0c9e26b1615 (DEFAULT_TABLE table=default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:31:35.908156 16884 raft_consensus.cc:3037] T ba0ed72a55d74ab5b8b22fba90848094 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:35.919508 16883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c115e29673747f8b0712d441ac993fb" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:36.038261 17143 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.006s sys 0.005s
I20260923 17:31:36.041720 18246 raft_consensus.cc:2726] T 254793351e5245af9a4fa1642cad5e82 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:36.050196 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 191292e60d0d46fb89b52390feb48f80 with delete_type TABLET_DATA_DELETED (replaced by tablet 18a117533c474766ad01c9a7bc3ef422) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:36.055636 17143 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.621s user 0.007s sys 0.006s
I20260923 17:31:35.859951 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:36.066969 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f70ce7043e46c396e9a0c9e26b1615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.075722 16861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8d088d9f2430e9c51f8ff62a860bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.078188 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:36.079697 16972 leader_election.cc:341] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:36.085476 18366 raft_consensus.cc:659] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260923 17:31:36.086221 18221 raft_consensus.cc:2258] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:36.090008 16869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473c1ab39b6542f989b35b72bea3b8b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.092877 16826 leader_election.cc:302] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c36e79694e154f84a46980719217d7f2; no voters: be93fda0843b44f5abc49e6f35c69217, e5c071cdf65042a59cad226a15a5532c
I20260923 17:31:36.108513 18221 ts_tablet_manager.cc:1905] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:36.115018 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7eaad1c706470da47819c5e373b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.110140 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:36.119020 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:36.122308 17143 tablet_bootstrap.cc:492] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:36.122866 17143 ts_tablet_manager.cc:1397] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 1.344s user 0.017s sys 0.006s
I20260923 17:31:36.129341 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:36.133208 17144 ts_tablet_manager.cc:1428] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Time spent starting tablet: real 0.551s user 0.002s sys 0.004s
W20260923 17:31:36.133524 17144 ts_tablet_manager.cc:1460] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Tablet startup took 551ms
I20260923 17:31:36.134699 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e]). Expected version 1 got 0
W20260923 17:31:36.133781 17144 ts_tablet_manager.cc:1462] T 070d38df3f0f43edab3826835c9503ce P be93fda0843b44f5abc49e6f35c69217: Trace:
0923 17:31:25.757384 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:25.757775 (+ 391us) service_pool.cc:227] Handling call
0923 17:31:25.758991 (+ 1216us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:25.759059 (+ 68us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:25.766899 (+ 7840us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:25.820124 (+ 53225us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:25.821963 (+ 1839us) inbound_call.cc:173] Queueing success response
0923 17:31:35.194381 (+9372418us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0923 17:31:35.581916 (+387535us) ts_tablet_manager.cc:1429] Starting tablet replica
0923 17:31:35.582480 (+ 564us) tablet_replica.cc:259] Starting instrumentation
0923 17:31:35.583002 (+ 522us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":982,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":324,"tablet-open.queue_time_us":9372480}
I20260923 17:31:36.135843 17144 tablet_bootstrap.cc:492] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217: Bootstrap starting.
I20260923 17:31:36.137442 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet b77ef164608f4ced8b1c2ecab6f88c1a (table default.Table_b0ce21ff6a6e46c8a4a37cdc3ab4f877 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 3 got 0
I20260923 17:31:36.140069 16388 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 070d38df3f0f43edab3826835c9503ce (table default.Table_04688becafca4883b1c56cbe6ee8132d [id=ef4aea9a69ae43e48cf913c554fcdac9]). Expected version 1 got 0
I20260923 17:31:36.141706 16387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: Sending DeleteTablet for 3 replicas of tablet d2796fcb72f5497a92208b40224739e9
I20260923 17:31:36.143291 16387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48560 (request call id 249) took 3664 ms. Trace:
I20260923 17:31:36.144081 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet d2796fcb72f5497a92208b40224739e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:34 UTC) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:36.143604 16387 rpcz_store.cc:276] 0923 17:31:32.479176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.421040 (+1941864us) service_pool.cc:227] Handling call
0923 17:31:34.465076 (+ 44036us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:34.465624 (+ 548us) catalog_manager.cc:2752] Modifying in-memory table state
0923 17:31:34.465760 (+ 136us) catalog_manager.cc:2761] Locking tablets
0923 17:31:34.466304 (+ 544us) catalog_manager.cc:2775] Removing table and tablets from system table
0923 17:31:34.552720 (+ 86416us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.552747 (+ 27us) write_op.cc:454] Released schema lock
0923 17:31:34.552976 (+ 229us) catalog_manager.cc:2793] Removing table from by-name map
0923 17:31:34.731251 (+178275us) catalog_manager.cc:2809] Committing in-memory state
0923 17:31:36.141475 (+1410224us) catalog_manager.cc:2813] Aborting table tasks
0923 17:31:36.143207 (+ 1732us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.537496 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.537877 (+ 381us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.537903 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:34.537917 (+ 14us) tablet.cc:660] Decoding operations
0923 17:31:34.538233 (+ 316us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.538275 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.538297 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0923 17:31:34.538578 (+ 281us) tablet.cc:699] Row locks acquired
0923 17:31:34.538594 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:34.538711 (+ 117us) write_op.cc:270] Start()
0923 17:31:34.538793 (+ 82us) write_op.cc:276] Timestamp: P: 1790184694538675 usec, L: 0
0923 17:31:34.538812 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.539304 (+ 492us) log.cc:842] Serialized 774 byte log entry
0923 17:31:34.547770 (+ 8466us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.548076 (+ 306us) write_op.cc:301] APPLY: starting
0923 17:31:34.548123 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.548241 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.548354 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.548595 (+ 241us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.548719 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0923 17:31:34.548760 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:34.548917 (+ 157us) log.cc:842] Serialized 29 byte log entry
0923 17:31:34.549067 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.549131 (+ 64us) write_op.cc:454] Released schema lock
0923 17:31:34.549159 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":169,"apply.run_cpu_time_us":1237,"apply.run_wall_time_us":1236,"mutex_wait_us":69235,"num_ops":2,"prepare.queue_time_us":69388,"prepare.run_cpu_time_us":2075,"prepare.run_wall_time_us":2075,"raft.queue_time_us":1305,"raft.run_cpu_time_us":733,"raft.run_wall_time_us":730,"replication_time_us":8891}]]}
I20260923 17:31:36.175827 17144 tablet_bootstrap.cc:654] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:36.182509 18381 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:42676 (request call id 307) took 1115 ms. Trace:
I20260923 17:31:36.183058 18381 rpcz_store.cc:276] 0923 17:31:35.067301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.067461 (+ 160us) service_pool.cc:227] Handling call
0923 17:31:36.182439 (+1114978us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:35.082043 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0923 17:31:35.082414 (+ 371us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0923 17:31:35.082442 (+ 28us) alter_schema_op.cc:65] Acquired schema lock
0923 17:31:35.082489 (+ 47us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0923 17:31:35.082623 (+ 134us) alter_schema_op.cc:126] START. Timestamp: 1790184695082522
0923 17:31:35.082644 (+ 21us) op_driver.cc:331] REPLICATION: starting
0923 17:31:35.083214 (+ 570us) log.cc:842] Serialized 173 byte log entry
0923 17:31:35.644840 (+561626us) op_driver.cc:444] REPLICATION: finished
0923 17:31:35.852321 (+207481us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0923 17:31:35.976906 (+124585us) log.cc:842] Serialized 5 byte log entry
0923 17:31:36.144003 (+167097us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.178134 (+ 34131us) log.cc:842] Serialized 14 byte log entry
0923 17:31:36.182296 (+ 4162us) alter_schema_op.cc:71] Released schema lock
0923 17:31:36.182323 (+ 27us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":207270,"prepare.queue_time_us":13659,"prepare.run_cpu_time_us":1823,"prepare.run_wall_time_us":68951,"replication_time_us":562104,"thread_start_us":328218,"threads_started":3,"wal-append.queue_time_us":63658}]]}
I20260923 17:31:36.191963 16386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_60e50e047c1244eab5a6b0552438404f [id=98651426b57f456a9a13bd9bc11a27f8] alter complete (version 3)
I20260923 17:31:36.197582 17144 tablet_bootstrap.cc:492] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217: No bootstrap required, opened a new log
I20260923 17:31:36.199194 17144 ts_tablet_manager.cc:1397] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217: Time spent bootstrapping tablet: real 0.064s user 0.011s sys 0.000s
I20260923 17:31:36.218849 16379 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d] alter complete (version 1)
I20260923 17:31:36.247421 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8d088d9f2430e9c51f8ff62a860bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.248248 17011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f70ce7043e46c396e9a0c9e26b1615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.248667 17146 tablet_bootstrap.cc:492] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:36.248935 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a117533c474766ad01c9a7bc3ef422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:36.253988 17146 tablet_bootstrap.cc:654] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:36.295670 17146 tablet_bootstrap.cc:492] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:36.296058 17146 ts_tablet_manager.cc:1397] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.013s
I20260923 17:31:36.315694 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet d2796fcb72f5497a92208b40224739e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:34 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:36.351828 17143 raft_consensus.cc:348] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:36.359936 17143 raft_consensus.cc:374] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:36.360339 17143 raft_consensus.cc:729] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:36.360985 17143 consensus_queue.cc:260] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:36.363505 17143 ts_tablet_manager.cc:1428] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.240s user 0.009s sys 0.000s
I20260923 17:31:36.381520 16386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_61c9bac7f4414016b97e990deaa211c5 [id=0160315adde14078a4216959ea769003] alter complete (version 1)
I20260923 17:31:36.384171 16865 tablet_service.cc:1514] Processing DeleteTablet for tablet f05a6f83314d433a9502e0c00ba8ad09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:20 UTC) from {username='slave'} at 127.0.0.1:36510
W20260923 17:31:36.386021 16386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.130:46845 (request call id 42) took 2482 ms (client timeout 1000 ms). Trace:
I20260923 17:31:36.394367 17143 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
W20260923 17:31:36.411582 16383 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 39) took 2181 ms (client timeout 1000 ms). Trace:
W20260923 17:31:36.414455 16383 rpcz_store.cc:269] 0923 17:31:34.228922 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.360620 (+131698us) service_pool.cc:227] Handling call
0923 17:31:36.410273 (+2049653us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18939,"reported_tablets":60}
I20260923 17:31:36.421866 18219 ts_tablet_manager.cc:1905] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:36.386400 16386 rpcz_store.cc:269] 0923 17:31:33.903919 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.304380 (+400461us) service_pool.cc:227] Handling call
0923 17:31:36.385925 (+2081545us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.951909 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.952415 (+ 506us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.952441 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:34.952456 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:34.952767 (+ 311us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.952811 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.952835 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:34.953002 (+ 167us) tablet.cc:699] Row locks acquired
0923 17:31:34.953019 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:34.953134 (+ 115us) write_op.cc:270] Start()
0923 17:31:34.953226 (+ 92us) write_op.cc:276] Timestamp: P: 1790184694953105 usec, L: 0
0923 17:31:34.953375 (+ 149us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.953939 (+ 564us) log.cc:842] Serialized 536 byte log entry
0923 17:31:35.027705 (+ 73766us) op_driver.cc:444] REPLICATION: finished
0923 17:31:35.027937 (+ 232us) write_op.cc:301] APPLY: starting
0923 17:31:35.027987 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:35.028083 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:35.028173 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:35.028406 (+ 233us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:35.028537 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:35.028576 (+ 39us) write_op.cc:312] APPLY: finished
0923 17:31:35.028773 (+ 197us) log.cc:842] Serialized 23 byte log entry
0923 17:31:35.029418 (+ 645us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.029501 (+ 83us) write_op.cc:454] Released schema lock
0923 17:31:35.029532 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:35.029713 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.029742 (+ 29us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:36.166986 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:36.167394 (+ 408us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:36.167420 (+ 26us) write_op.cc:435] Acquired schema lock
0923 17:31:36.167436 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:36.167737 (+ 301us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:36.167781 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:36.167805 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:36.167974 (+ 169us) tablet.cc:699] Row locks acquired
0923 17:31:36.167986 (+ 12us) write_op.cc:260] PREPARE: finished
0923 17:31:36.168107 (+ 121us) write_op.cc:270] Start()
0923 17:31:36.168196 (+ 89us) write_op.cc:276] Timestamp: P: 1790184696168059 usec, L: 0
0923 17:31:36.168214 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:36.168790 (+ 576us) log.cc:842] Serialized 356 byte log entry
0923 17:31:36.189625 (+ 20835us) op_driver.cc:444] REPLICATION: finished
0923 17:31:36.190012 (+ 387us) write_op.cc:301] APPLY: starting
0923 17:31:36.190075 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:36.190201 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:36.190299 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:36.190534 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:36.190705 (+ 171us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:36.190754 (+ 49us) write_op.cc:312] APPLY: finished
0923 17:31:36.190995 (+ 241us) log.cc:842] Serialized 23 byte log entry
0923 17:31:36.191182 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.191257 (+ 75us) write_op.cc:454] Released schema lock
0923 17:31:36.191291 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:36.191530 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.191555 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:36.327050 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:36.327490 (+ 440us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:36.327520 (+ 30us) write_op.cc:435] Acquired schema lock
0923 17:31:36.327536 (+ 16us) tablet.cc:660] Decoding operations
0923 17:31:36.327815 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:36.327865 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:36.327889 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:36.328038 (+ 149us) tablet.cc:699] Row locks acquired
0923 17:31:36.328054 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:36.328169 (+ 115us) write_op.cc:270] Start()
0923 17:31:36.328284 (+ 115us) write_op.cc:276] Timestamp: P: 1790184696328116 usec, L: 0
0923 17:31:36.328316 (+ 32us) op_driver.cc:331] REPLICATION: starting
0923 17:31:36.328805 (+ 489us) log.cc:842] Serialized 356 byte log entry
0923 17:31:36.372191 (+ 43386us) op_driver.cc:444] REPLICATION: finished
0923 17:31:36.377821 (+ 5630us) write_op.cc:301] APPLY: starting
0923 17:31:36.377881 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:36.377983 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:36.378088 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:36.378307 (+ 219us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:36.378452 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:36.378496 (+ 44us) write_op.cc:312] APPLY: finished
0923 17:31:36.378724 (+ 228us) log.cc:842] Serialized 23 byte log entry
0923 17:31:36.381053 (+ 2329us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.381125 (+ 72us) write_op.cc:454] Released schema lock
0923 17:31:36.381159 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:36.381344 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.381366 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":1446,"apply.run_wall_time_us":1954,"num_ops":1,"prepare.queue_time_us":39302,"prepare.run_cpu_time_us":2258,"prepare.run_wall_time_us":153798,"raft.queue_time_us":25724,"raft.run_cpu_time_us":983,"raft.run_wall_time_us":1314,"replication_time_us":74231}],["op",{"apply.queue_time_us":207,"apply.run_cpu_time_us":1715,"apply.run_wall_time_us":1715,"num_ops":1,"prepare.queue_time_us":254,"prepare.run_cpu_time_us":2133,"prepare.run_wall_time_us":2165,"raft.queue_time_us":3342,"raft.run_cpu_time_us":1376,"raft.run_wall_time_us":1377,"replication_time_us":21267}],["op",{"apply.queue_time_us":554,"apply.run_cpu_time_us":1586,"apply.run_wall_time_us":3715,"num_ops":1,"prepare.queue_time_us":332,"prepare.run_cpu_time_us":2083,"prepare.run_wall_time_us":2084,"raft.queue_time_us":3508,"raft.run_cpu_time_us":1414,"raft.run_wall_time_us":1413,"replication_time_us":43781}]]}
I20260923 17:31:36.448449 18221 ts_tablet_manager.cc:1918] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:36.449050 18221 log.cc:1196] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/261d1d03290546969a74ca60beb890eb
I20260923 17:31:36.450127 18221 ts_tablet_manager.cc:1939] T 261d1d03290546969a74ca60beb890eb P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:36.453660 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 237) took 4838 ms. Trace:
I20260923 17:31:36.454129 18221 rpcz_store.cc:276] 0923 17:31:31.613796 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:31.622808 (+ 9012us) service_pool.cc:227] Handling call
0923 17:31:35.456350 (+3833542us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:35.456395 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:36.330545 (+874150us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.448100 (+117555us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.452456 (+ 4356us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25431,"tablet-delete.queue_time_us":3832646}
I20260923 17:31:36.456033 18221 tablet_replica.cc:330] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:36.456157 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 261d1d03290546969a74ca60beb890eb (table default.Table_76bda21bbdc644518bea95216cd34ea7 [id=b8d6e55b1ee54de3ab438e37a3ff44f5]) successfully deleted
W20260923 17:31:36.498519 16372 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c consensus state has no leader
W20260923 17:31:36.503306 16828 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.508s user 0.009s sys 0.012s
W20260923 17:31:36.503674 16828 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.508s user 0.009s sys 0.012s
I20260923 17:31:36.520138 17144 raft_consensus.cc:348] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.520889 17144 raft_consensus.cc:374] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:36.521147 17144 raft_consensus.cc:729] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be93fda0843b44f5abc49e6f35c69217, State: Initialized, Role: FOLLOWER
I20260923 17:31:36.521759 17144 consensus_queue.cc:260] T c3d4abafcef943c08ae0755bae06f32c P be93fda0843b44f5abc49e6f35c69217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.541589 18366 raft_consensus.cc:2726] T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:36.471629 17146 raft_consensus.cc:348] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.568470 17146 raft_consensus.cc:374] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:36.568763 17146 raft_consensus.cc:729] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:36.569367 17146 consensus_queue.cc:260] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.571296 17146 ts_tablet_manager.cc:1428] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.275s user 0.004s sys 0.000s
I20260923 17:31:36.574584 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feebaa76bcd4ff5b707814191cb3341" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:36.471909 18396 tablet.cc:1722] T 6a90f7348a964a7f871e93dde7ab82f2 P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:36.474551 18386 raft_consensus.cc:482] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:36.475462 17011 tablet_service.cc:1514] Processing DeleteTablet for tablet 456273cf57314cb8a05c8e765543913f with delete_type TABLET_DATA_DELETED (replaced by tablet 3fc89f029c1c4d8fb1b192017cade4f7) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:36.475929 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feebaa76bcd4ff5b707814191cb3341 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c83942d30a543d692641024253078cc) from {username='slave'} at 127.0.0.1:38390
I20260923 17:31:36.508771 17026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:36.578585 18386 raft_consensus.cc:504] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.579021 17026 raft_consensus.cc:2454] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:36.510977 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b11a6fd84840178b81d2b7610c7351" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:36.579877 17038 raft_consensus.cc:2454] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be93fda0843b44f5abc49e6f35c69217 in term 0.
I20260923 17:31:36.517343 17040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:36.580950 18386 leader_election.cc:288] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:36.583246 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b11a6fd84840178b81d2b7610c7351" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:36.593376 16972 leader_election.cc:341] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
W20260923 17:31:36.601774 18397 log.cc:924] Time spent T c1d71b22338241839b184e50d7fdd8f6 P c36e79694e154f84a46980719217d7f2: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
I20260923 17:31:36.622975 16382 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 253) took 3834 ms. Trace:
I20260923 17:31:36.625420 16892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.15.246.129:46475 (request call id 590) took 1500 ms. Trace:
I20260923 17:31:36.623317 16382 rpcz_store.cc:276] 0923 17:31:32.788842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.554257 (+1765415us) service_pool.cc:227] Handling call
0923 17:31:34.770071 (+215814us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:34.770740 (+ 669us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:34.771266 (+ 526us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:34.771486 (+ 220us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:34.804289 (+ 32803us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.804317 (+ 28us) write_op.cc:454] Released schema lock
0923 17:31:34.804541 (+ 224us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:36.622789 (+1818248us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.784054 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.784559 (+ 505us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.784588 (+ 29us) write_op.cc:435] Acquired schema lock
0923 17:31:34.784606 (+ 18us) tablet.cc:660] Decoding operations
0923 17:31:34.784885 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.784930 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.784954 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:34.785110 (+ 156us) tablet.cc:699] Row locks acquired
0923 17:31:34.785127 (+ 17us) write_op.cc:260] PREPARE: finished
0923 17:31:34.785239 (+ 112us) write_op.cc:270] Start()
0923 17:31:34.785334 (+ 95us) write_op.cc:276] Timestamp: P: 1790184694785185 usec, L: 0
0923 17:31:34.785352 (+ 18us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.785837 (+ 485us) log.cc:842] Serialized 405 byte log entry
0923 17:31:34.794341 (+ 8504us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.794606 (+ 265us) write_op.cc:301] APPLY: starting
0923 17:31:34.794655 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.794738 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.794859 (+ 121us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.794989 (+ 130us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.795107 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:34.795155 (+ 48us) write_op.cc:312] APPLY: finished
0923 17:31:34.795305 (+ 150us) log.cc:842] Serialized 23 byte log entry
0923 17:31:34.795445 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.795499 (+ 54us) write_op.cc:454] Released schema lock
0923 17:31:34.795530 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":138,"apply.run_cpu_time_us":1082,"apply.run_wall_time_us":1082,"num_ops":1,"prepare.queue_time_us":622,"prepare.run_cpu_time_us":2128,"prepare.run_wall_time_us":5645,"raft.queue_time_us":81,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":589,"replication_time_us":8916}]]}
I20260923 17:31:36.627082 16384 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48560 (request call id 252) took 3923 ms. Trace:
I20260923 17:31:36.627785 16346 master-stress-test.cc:580] Killed test thread 5/10
I20260923 17:31:36.628051 16346 master-stress-test.cc:578] Killing test thread 6/10
I20260923 17:31:36.627394 16384 rpcz_store.cc:276] 0923 17:31:32.703522 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.498511 (+1794989us) service_pool.cc:227] Handling call
0923 17:31:34.731411 (+232900us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0923 17:31:34.739149 (+ 7738us) catalog_manager.cc:3616] Apply alter schema
0923 17:31:34.739701 (+ 552us) catalog_manager.cc:3639] Acquired catalog manager lock
0923 17:31:34.739902 (+ 201us) catalog_manager.cc:3786] Updating metadata on disk
0923 17:31:34.775420 (+ 35518us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.775445 (+ 25us) write_op.cc:454] Released schema lock
0923 17:31:34.775832 (+ 387us) catalog_manager.cc:3817] Committing alterations to in-memory state
0923 17:31:36.626925 (+1851093us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:34.741067 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:34.741462 (+ 395us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:34.741487 (+ 25us) write_op.cc:435] Acquired schema lock
0923 17:31:34.741498 (+ 11us) tablet.cc:660] Decoding operations
0923 17:31:34.741770 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:34.741815 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:34.741837 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:34.741987 (+ 150us) tablet.cc:699] Row locks acquired
0923 17:31:34.742000 (+ 13us) write_op.cc:260] PREPARE: finished
0923 17:31:34.742115 (+ 115us) write_op.cc:270] Start()
0923 17:31:34.742205 (+ 90us) write_op.cc:276] Timestamp: P: 1790184694742062 usec, L: 0
0923 17:31:34.742221 (+ 16us) op_driver.cc:331] REPLICATION: starting
0923 17:31:34.742707 (+ 486us) log.cc:842] Serialized 405 byte log entry
0923 17:31:34.761425 (+ 18718us) op_driver.cc:444] REPLICATION: finished
0923 17:31:34.761817 (+ 392us) write_op.cc:301] APPLY: starting
0923 17:31:34.761924 (+ 107us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:34.762089 (+ 165us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:34.762208 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:34.762355 (+ 147us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:34.762475 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:34.762516 (+ 41us) write_op.cc:312] APPLY: finished
0923 17:31:34.762679 (+ 163us) log.cc:842] Serialized 23 byte log entry
0923 17:31:34.762806 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:34.762860 (+ 54us) write_op.cc:454] Released schema lock
0923 17:31:34.762894 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":7640,"child_traces":[["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":1267,"apply.run_wall_time_us":1267,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":1927,"prepare.run_wall_time_us":4386,"raft.queue_time_us":78,"raft.run_cpu_time_us":613,"raft.run_wall_time_us":623,"replication_time_us":19077}]]}
I20260923 17:31:36.631651 16346 master-stress-test.cc:580] Killed test thread 6/10
I20260923 17:31:36.631944 16346 master-stress-test.cc:578] Killing test thread 7/10
I20260923 17:31:36.632468 16346 master-stress-test.cc:580] Killed test thread 7/10
I20260923 17:31:36.632674 16346 master-stress-test.cc:578] Killing test thread 8/10
I20260923 17:31:36.632988 16346 master-stress-test.cc:580] Killed test thread 8/10
I20260923 17:31:36.633165 16346 master-stress-test.cc:578] Killing test thread 9/10
I20260923 17:31:36.633443 16346 master-stress-test.cc:580] Killed test thread 9/10
I20260923 17:31:36.633625 16346 master-stress-test.cc:578] Killing test thread 10/10
I20260923 17:31:36.633901 16346 master-stress-test.cc:580] Killed test thread 10/10
I20260923 17:31:36.634291 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16630
I20260923 17:31:36.583683 17143 tablet_bootstrap.cc:700] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: Time spent opening tablet: real 0.117s user 0.001s sys 0.001s
I20260923 17:31:36.640785 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "354f66c852b94b1d9f44b6bc012cf81b" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:36.642498 16887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84e322a4de24cbabf450fceb098d6e7" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:36.641008 16883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b11a6fd84840178b81d2b7610c7351" candidate_uuid: "be93fda0843b44f5abc49e6f35c69217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:36.651189 16388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.131:54045 (request call id 40) took 1044 ms (client timeout 1000 ms). Trace:
W20260923 17:31:36.651553 16388 rpcz_store.cc:269] 0923 17:31:35.607232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.607535 (+ 303us) service_pool.cc:227] Handling call
0923 17:31:36.651109 (+1043574us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":230723,"reported_tablets":63}
I20260923 17:31:36.664470 17143 tablet_bootstrap.cc:654] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:36.679396 18255 raft_consensus.cc:482] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:36.681588 18255 raft_consensus.cc:504] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.683733 18255 leader_election.cc:288] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:36.691560 16385 catalog_manager.cc:5495] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P be93fda0843b44f5abc49e6f35c69217 reported cstate change: term changed from 0 to 1, leader changed from <none> to be93fda0843b44f5abc49e6f35c69217 (127.15.246.129). New cstate: current_term: 1 leader_uuid: "be93fda0843b44f5abc49e6f35c69217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } health_report { overall_health: UNKNOWN } } }
I20260923 17:31:36.693455 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:36.695056 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (table default.Table_c1bfc0841ebe4015841fdddeeae9b6e9 [id=8abd6be5200d427c80760d3772ccabce]). Expected version 1 got 0
I20260923 17:31:36.698068 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (table default.Table_3f9064b9a5b146cd89d2ec54efa1fa4b [id=40ff0e1afb354d248c9307efd7eaabdd]). Expected version 1 got 0
I20260923 17:31:36.699954 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:36.701519 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
W20260923 17:31:36.712495 16828 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.15.246.189:45151, user_credentials={real_user=slave}} blocked reactor thread for 68140.9us
I20260923 17:31:36.715119 18410 tablet.cc:1722] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:36.625964 16892 rpcz_store.cc:276] 0923 17:31:35.125194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.125291 (+ 97us) service_pool.cc:227] Handling call
0923 17:31:35.125647 (+ 356us) raft_consensus.cc:1457] Updating replica for 1 ops
0923 17:31:35.414851 (+289204us) raft_consensus.cc:1520] Early marking committed up to index 0
0923 17:31:35.420668 (+ 5817us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0923 17:31:36.085171 (+664503us) log.cc:842] Serialized 26 byte log entry
0923 17:31:36.524090 (+438919us) raft_consensus.cc:1631] Marking committed up to 1
0923 17:31:36.524762 (+ 672us) log.cc:842] Serialized 14 byte log entry
0923 17:31:36.533970 (+ 9208us) raft_consensus.cc:1683] Filling consensus response to leader.
0923 17:31:36.595565 (+ 61595us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54000438d4. stack: 00007fe3d2037033 00007fe3d2036f30 00007fe3d1bb8534 00000000004db2b4 00000000004db249 00007fe3d8f43471 00007fe3d8fbef51 00007fe3d8fbd99e 00007fe3d9f9a2e9 00007fe3d8770684 00007fe3d8770612 00007fe3d8770541 00007fe3d87704bc 00007fe3d876f772 00007fe3d6d6d94c 00007fe3d6d6cd86
0923 17:31:36.595623 (+ 58us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0923 17:31:36.624929 (+ 29306us) raft_consensus.cc:1671] finished
0923 17:31:36.625047 (+ 118us) raft_consensus.cc:1677] UpdateReplicas() finished
0923 17:31:36.625211 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":206,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":64789,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":280,"spinlock_wait_cycles":386704128,"thread_start_us":299551,"threads_started":2,"wal-append.queue_time_us":215678}
I20260923 17:31:36.744051 17143 tablet_bootstrap.cc:492] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:36.744552 17143 ts_tablet_manager.cc:1397] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.374s user 0.008s sys 0.004s
I20260923 17:31:36.747061 17143 raft_consensus.cc:348] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.747716 17143 raft_consensus.cc:374] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:36.747933 17143 raft_consensus.cc:729] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:36.748790 17143 consensus_queue.cc:260] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.750315 17143 ts_tablet_manager.cc:1428] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 17:31:36.752902 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e]). Expected version 1 got 0
I20260923 17:31:36.753690 16385 catalog_manager.cc:5587] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
I20260923 17:31:36.774909 18221 raft_consensus.cc:2229] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:36.775476 18221 raft_consensus.cc:2258] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:36.777587 18221 ts_tablet_manager.cc:1905] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:36.809731 17143 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:36.831751 17143 tablet_bootstrap.cc:654] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:36.840422 16379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 46) took 2744 ms (client timeout 1000 ms). Trace:
I20260923 17:31:36.863888 17143 tablet_bootstrap.cc:492] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:36.864491 17143 ts_tablet_manager.cc:1397] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.004s
W20260923 17:31:36.840750 16379 rpcz_store.cc:269] 0923 17:31:34.096507 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.326785 (+230278us) service_pool.cc:227] Handling call
0923 17:31:36.840351 (+2513566us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:35.741664 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:35.742192 (+ 528us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:35.742239 (+ 47us) write_op.cc:435] Acquired schema lock
0923 17:31:35.742268 (+ 29us) tablet.cc:660] Decoding operations
0923 17:31:35.742580 (+ 312us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:35.742624 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:35.742648 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:35.742796 (+ 148us) tablet.cc:699] Row locks acquired
0923 17:31:35.742812 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:35.742935 (+ 123us) write_op.cc:270] Start()
0923 17:31:35.743029 (+ 94us) write_op.cc:276] Timestamp: P: 1790184695742880 usec, L: 0
0923 17:31:35.743043 (+ 14us) op_driver.cc:331] REPLICATION: starting
0923 17:31:35.743604 (+ 561us) log.cc:842] Serialized 536 byte log entry
0923 17:31:35.752860 (+ 9256us) op_driver.cc:444] REPLICATION: finished
0923 17:31:35.753148 (+ 288us) write_op.cc:301] APPLY: starting
0923 17:31:35.753201 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:35.753287 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:35.753376 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:35.753554 (+ 178us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:35.753671 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:35.753711 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:35.753917 (+ 206us) log.cc:842] Serialized 23 byte log entry
0923 17:31:35.754063 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.754117 (+ 54us) write_op.cc:454] Released schema lock
0923 17:31:35.754149 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:35.754327 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:35.754349 (+ 22us) write_op.cc:454] Released schema lock
Related trace 'op':
0923 17:31:36.172733 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:36.173155 (+ 422us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:36.173179 (+ 24us) write_op.cc:435] Acquired schema lock
0923 17:31:36.173194 (+ 15us) tablet.cc:660] Decoding operations
0923 17:31:36.173460 (+ 266us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:36.173499 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:36.173525 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:36.173783 (+ 258us) tablet.cc:699] Row locks acquired
0923 17:31:36.173814 (+ 31us) write_op.cc:260] PREPARE: finished
0923 17:31:36.173933 (+ 119us) write_op.cc:270] Start()
0923 17:31:36.174022 (+ 89us) write_op.cc:276] Timestamp: P: 1790184696173889 usec, L: 0
0923 17:31:36.174046 (+ 24us) op_driver.cc:331] REPLICATION: starting
0923 17:31:36.174514 (+ 468us) log.cc:842] Serialized 356 byte log entry
0923 17:31:36.209979 (+ 35465us) op_driver.cc:444] REPLICATION: finished
0923 17:31:36.213358 (+ 3379us) write_op.cc:301] APPLY: starting
0923 17:31:36.213427 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:36.213530 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:36.213633 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:36.213920 (+ 287us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:36.214086 (+ 166us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:36.214134 (+ 48us) write_op.cc:312] APPLY: finished
0923 17:31:36.214355 (+ 221us) log.cc:842] Serialized 23 byte log entry
0923 17:31:36.218253 (+ 3898us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.218344 (+ 91us) write_op.cc:454] Released schema lock
0923 17:31:36.218384 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0923 17:31:36.219550 (+ 1166us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.219583 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54709,"reported_tablets":58,"child_traces":[["op",{"apply.queue_time_us":144,"apply.run_cpu_time_us":1354,"apply.run_wall_time_us":1353,"num_ops":1,"prepare.queue_time_us":200,"prepare.run_cpu_time_us":2296,"prepare.run_wall_time_us":2297,"raft.queue_time_us":3421,"raft.run_cpu_time_us":1416,"raft.run_wall_time_us":1415,"replication_time_us":9722}],["op",{"apply.queue_time_us":3156,"apply.run_cpu_time_us":1887,"apply.run_wall_time_us":6435,"num_ops":1,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":1915,"prepare.run_wall_time_us":3086,"replication_time_us":35844}]]}
I20260923 17:31:36.870765 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (table default.Table_c1bfc0841ebe4015841fdddeeae9b6e9 [id=8abd6be5200d427c80760d3772ccabce]). Expected version 1 got 0
I20260923 17:31:36.871506 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (table default.Table_3f9064b9a5b146cd89d2ec54efa1fa4b [id=40ff0e1afb354d248c9307efd7eaabdd]). Expected version 1 got 0
I20260923 17:31:36.881626 17143 raft_consensus.cc:348] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.882431 17143 raft_consensus.cc:374] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:36.882727 17143 raft_consensus.cc:729] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:36.883517 17143 consensus_queue.cc:260] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.885903 17143 ts_tablet_manager.cc:1428] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.021s user 0.004s sys 0.001s
I20260923 17:31:36.901510 17143 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:36.917011 17008 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38390 (request call id 267) took 1055 ms. Trace:
I20260923 17:31:36.917748 17008 rpcz_store.cc:276] 0923 17:31:35.861319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.861459 (+ 140us) service_pool.cc:227] Handling call
0923 17:31:36.173491 (+312032us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5800001898. stack: 00007f9e060c6033 00007f9e060c5f30 00007f9e05c47534 00000000004db2b4 00000000004db249 00007f9e05f3cf11 00007f9e060c0b7c 00007f9e060c073b 00007f9e0ef57b09 00007f9e0dfb4730 00007f9e0e096d21 00007f9e0e0741e3 00007f9e0e025869 00007f9e0c717c44 00007f9e0c717bd2 00007f9e0c717b01
0923 17:31:36.248527 (+ 75036us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0923 17:31:36.248629 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0923 17:31:36.423391 (+174762us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.608027 (+184636us) tablet_replica.cc:205] Creating consensus instance
0923 17:31:36.916607 (+308580us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":37014400}
I20260923 17:31:36.902035 18221 ts_tablet_manager.cc:1918] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:36.905784 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11123f6ee880405ea76479c80fcea7e7" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
I20260923 17:31:36.908133 18391 raft_consensus.cc:482] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:36.921546 18391 raft_consensus.cc:504] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } }
I20260923 17:31:36.924595 18391 leader_election.cc:288] T cfa712f9362e416e9cc6defcd8198bcd P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:36.927045 16972 leader_election.cc:341] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:36.928176 18221 log.cc:1196] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:36.929368 18221 ts_tablet_manager.cc:1939] T c48bda995b284cf6be7bb6f348ec0fd0 P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:36.934446 17146 tablet_bootstrap.cc:492] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:36.873041 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 7dbd8eb0ab9243bd81c3d6a893f18aea (table default.Table_038736374c594b9e8129ba1c363b280a [id=3acfda57412f48e4a9561d2b4cf7f98d]). Expected version 1 got 0
I20260923 17:31:36.946918 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:36.949198 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:36.952756 18414 raft_consensus.cc:482] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:36.958969 17143 tablet_bootstrap.cc:654] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:36.960050 18391 raft_consensus.cc:482] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:36.960534 18391 raft_consensus.cc:504] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.962618 18391 leader_election.cc:288] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:36.957401 18414 raft_consensus.cc:504] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:36.970501 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 238) took 4790 ms. Trace:
I20260923 17:31:36.970882 18221 rpcz_store.cc:276] 0923 17:31:32.180291 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.182062 (+ 1771us) service_pool.cc:227] Handling call
0923 17:31:36.455808 (+4273746us) ts_tablet_manager.cc:1145] Deleting tablet c48bda995b284cf6be7bb6f348ec0fd0
0923 17:31:36.455900 (+ 92us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:36.835837 (+379937us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.895874 (+ 60037us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:36.970431 (+ 74557us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":317609,"tablet-delete.queue_time_us":4273111}
I20260923 17:31:36.972695 18221 tablet_replica.cc:330] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:36.973748 18221 raft_consensus.cc:2229] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:36.974469 18221 raft_consensus.cc:2258] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:36.974637 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet c48bda995b284cf6be7bb6f348ec0fd0 (table default.Table_219bd5224fe64024867dec9b8bef6824 [id=fdcba39110ba4da0809ed11b2bcbdf8d]) successfully deleted
I20260923 17:31:36.979039 18414 leader_election.cc:288] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433)
I20260923 17:31:36.980423 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa712f9362e416e9cc6defcd8198bcd" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:36.998801 17146 tablet_bootstrap.cc:654] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:37.006409 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a992ad5280444dda64c1325da92a707" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:37.008987 16972 leader_election.cc:341] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:37.014009 18221 ts_tablet_manager.cc:1905] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:37.015530 16385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.15.246.129:58511 (request call id 47) took 1367 ms (client timeout 1000 ms). Trace:
I20260923 17:31:37.020460 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 6a90f7348a964a7f871e93dde7ab82f2 (table default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2]). Expected version 1 got 0
W20260923 17:31:37.020987 16827 leader_election.cc:341] T 7c115e29673747f8b0712d441ac993fb P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): Not found: Tablet not found: 7c115e29673747f8b0712d441ac993fb
W20260923 17:31:37.017140 16385 rpcz_store.cc:269] 0923 17:31:35.647852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.648113 (+ 261us) service_pool.cc:227] Handling call
0923 17:31:36.785512 (+1137399us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.785538 (+ 26us) write_op.cc:454] Released schema lock
0923 17:31:37.015459 (+229921us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0923 17:31:36.758064 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0923 17:31:36.758585 (+ 521us) write_op.cc:432] Acquiring schema lock in shared mode
0923 17:31:36.758616 (+ 31us) write_op.cc:435] Acquired schema lock
0923 17:31:36.758634 (+ 18us) tablet.cc:660] Decoding operations
0923 17:31:36.758961 (+ 327us) write_op.cc:620] Acquiring the partition lock for write op
0923 17:31:36.759009 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0923 17:31:36.759041 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0923 17:31:36.759225 (+ 184us) tablet.cc:699] Row locks acquired
0923 17:31:36.759241 (+ 16us) write_op.cc:260] PREPARE: finished
0923 17:31:36.759365 (+ 124us) write_op.cc:270] Start()
0923 17:31:36.759457 (+ 92us) write_op.cc:276] Timestamp: P: 1790184696759322 usec, L: 0
0923 17:31:36.759476 (+ 19us) op_driver.cc:331] REPLICATION: starting
0923 17:31:36.760021 (+ 545us) log.cc:842] Serialized 536 byte log entry
0923 17:31:36.769827 (+ 9806us) op_driver.cc:444] REPLICATION: finished
0923 17:31:36.770775 (+ 948us) write_op.cc:301] APPLY: starting
0923 17:31:36.770840 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0923 17:31:36.770934 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0923 17:31:36.771039 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0923 17:31:36.771272 (+ 233us) tablet.cc:1381] finished ApplyRowOperation cycle
0923 17:31:36.771398 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0923 17:31:36.771438 (+ 40us) write_op.cc:312] APPLY: finished
0923 17:31:36.771626 (+ 188us) log.cc:842] Serialized 23 byte log entry
0923 17:31:36.771775 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0923 17:31:36.771847 (+ 72us) write_op.cc:454] Released schema lock
0923 17:31:36.771883 (+ 36us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":88844,"reported_tablets":58,"child_traces":[["op",{"apply.queue_time_us":786,"apply.run_cpu_time_us":1300,"apply.run_wall_time_us":1299,"num_ops":1,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":2351,"prepare.run_wall_time_us":2352,"raft.queue_time_us":3389,"raft.run_cpu_time_us":1488,"raft.run_wall_time_us":1487,"replication_time_us":10264}]]}
I20260923 17:31:37.007617 18219 ts_tablet_manager.cc:1918] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:37.028432 18219 log.cc:1196] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/db3504813e9c4698a546b6d6fa0fbcb6
I20260923 17:31:37.029562 18219 ts_tablet_manager.cc:1939] T db3504813e9c4698a546b6d6fa0fbcb6 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:37.035751 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 247) took 2808 ms. Trace:
I20260923 17:31:37.036540 18371 raft_consensus.cc:482] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:37.037077 18371 raft_consensus.cc:504] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.039670 18371 leader_election.cc:288] T 523963a1d7fb4ac0b4c9fb3ddde7fe6c P c36e79694e154f84a46980719217d7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879), e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559)
I20260923 17:31:37.042014 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523963a1d7fb4ac0b4c9fb3ddde7fe6c" candidate_uuid: "c36e79694e154f84a46980719217d7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c071cdf65042a59cad226a15a5532c" is_pre_election: true
I20260923 17:31:37.044401 18219 rpcz_store.cc:276] 0923 17:31:34.227616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.228465 (+ 849us) service_pool.cc:227] Handling call
0923 17:31:35.637329 (+1408864us) ts_tablet_manager.cc:1145] Deleting tablet db3504813e9c4698a546b6d6fa0fbcb6
0923 17:31:35.715016 (+ 77687us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:36.761223 (+1046207us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.007248 (+246025us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.035677 (+ 28429us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1408275}
I20260923 17:31:37.046329 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 248) took 2778 ms. Trace:
I20260923 17:31:37.046638 18219 rpcz_store.cc:276] 0923 17:31:34.268417 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.268612 (+ 195us) service_pool.cc:227] Handling call
0923 17:31:37.045961 (+2777349us) ts_tablet_manager.cc:1145] Deleting tablet 006895667f324d049debbebb132ed09b
0923 17:31:37.046048 (+ 87us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.046274 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2776732}
I20260923 17:31:37.047678 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 249) took 2777 ms. Trace:
I20260923 17:31:37.048695 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet db3504813e9c4698a546b6d6fa0fbcb6 (table default.Table_206108d7a4384c5e8a3201f96c3cb705 [id=004f88ba43bd4f17ace3805b35d3527d]) successfully deleted
W20260923 17:31:37.049641 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 006895667f324d049debbebb132ed09b because the tablet was not found. No further retry: Not found: Tablet not found: 006895667f324d049debbebb132ed09b
I20260923 17:31:37.047950 18219 rpcz_store.cc:276] 0923 17:31:34.270320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.270454 (+ 134us) service_pool.cc:227] Handling call
0923 17:31:37.047365 (+2776911us) ts_tablet_manager.cc:1145] Deleting tablet 5c929b8beaaf4c7aa668852b63af2204
0923 17:31:37.047422 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.047627 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2776425}
W20260923 17:31:37.058260 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 5c929b8beaaf4c7aa668852b63af2204 because the tablet was not found. No further retry: Not found: Tablet not found: 5c929b8beaaf4c7aa668852b63af2204
I20260923 17:31:37.059717 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 250) took 2787 ms. Trace:
I20260923 17:31:37.062505 18413 tablet.cc:1722] T bd53fd73fce64b658b09a0fbfcdbb0fd P c36e79694e154f84a46980719217d7f2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260923 17:31:37.067760 18219 rpcz_store.cc:276] 0923 17:31:34.272562 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.272677 (+ 115us) service_pool.cc:227] Handling call
0923 17:31:37.059400 (+2786723us) ts_tablet_manager.cc:1145] Deleting tablet 19ebfb3cfd3d49b39c7cf453199ac8bd
0923 17:31:37.059462 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.059667 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2786219}
I20260923 17:31:37.069089 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 255) took 2746 ms. Trace:
W20260923 17:31:37.069284 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 19ebfb3cfd3d49b39c7cf453199ac8bd because the tablet was not found. No further retry: Not found: Tablet not found: 19ebfb3cfd3d49b39c7cf453199ac8bd
I20260923 17:31:37.071314 18219 rpcz_store.cc:276] 0923 17:31:34.322289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.322462 (+ 173us) service_pool.cc:227] Handling call
0923 17:31:37.068725 (+2746263us) ts_tablet_manager.cc:1145] Deleting tablet 261d1d03290546969a74ca60beb890eb
0923 17:31:37.068783 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.069033 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2744683}
W20260923 17:31:37.073272 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 261d1d03290546969a74ca60beb890eb because the tablet was not found. No further retry: Not found: Tablet not found: 261d1d03290546969a74ca60beb890eb
I20260923 17:31:37.074785 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 252) took 2767 ms. Trace:
I20260923 17:31:37.076210 18219 rpcz_store.cc:276] 0923 17:31:34.307555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.307695 (+ 140us) service_pool.cc:227] Handling call
0923 17:31:37.074425 (+2766730us) ts_tablet_manager.cc:1145] Deleting tablet 8e7c8e360d954ce78913e78c8662c341
0923 17:31:37.074490 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.074729 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2750081}
W20260923 17:31:37.078169 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 8e7c8e360d954ce78913e78c8662c341 because the tablet was not found. No further retry: Not found: Tablet not found: 8e7c8e360d954ce78913e78c8662c341
I20260923 17:31:37.079439 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 254) took 2757 ms. Trace:
I20260923 17:31:37.081357 18219 rpcz_store.cc:276] 0923 17:31:34.321840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.323213 (+ 1373us) service_pool.cc:227] Handling call
0923 17:31:37.079148 (+2755935us) ts_tablet_manager.cc:1145] Deleting tablet 7c115e29673747f8b0712d441ac993fb
0923 17:31:37.079201 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.079395 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2753749}
W20260923 17:31:37.083324 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet 7c115e29673747f8b0712d441ac993fb because the tablet was not found. No further retry: Not found: Tablet not found: 7c115e29673747f8b0712d441ac993fb
I20260923 17:31:37.086138 17143 tablet_bootstrap.cc:492] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2: No bootstrap required, opened a new log
I20260923 17:31:37.086606 17143 ts_tablet_manager.cc:1397] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2: Time spent bootstrapping tablet: real 0.185s user 0.007s sys 0.003s
I20260923 17:31:37.087209 18219 tablet_replica.cc:330] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:37.088718 18219 raft_consensus.cc:2229] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:37.089393 18219 pending_rounds.cc:62] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Trying to abort 1 pending ops.
I20260923 17:31:37.089673 18219 pending_rounds.cc:68] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7332596508422770688 op_type: NO_OP noop_request { }
I20260923 17:31:37.089849 17143 raft_consensus.cc:348] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.090211 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:37.090516 17143 raft_consensus.cc:374] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:37.090756 17143 raft_consensus.cc:729] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36e79694e154f84a46980719217d7f2, State: Initialized, Role: FOLLOWER
I20260923 17:31:37.090728 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 6dbfc01e66404a5da4f30c5eff8e7f45 (table default.Table_3f9064b9a5b146cd89d2ec54efa1fa4b [id=40ff0e1afb354d248c9307efd7eaabdd]). Expected version 1 got 0
I20260923 17:31:37.091423 17143 consensus_queue.cc:260] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.090830 18219 raft_consensus.cc:2866] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260923 17:31:37.094344 18219 raft_consensus.cc:2258] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:37.096446 18219 ts_tablet_manager.cc:1905] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:37.103737 17146 tablet_bootstrap.cc:492] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:37.104172 17146 ts_tablet_manager.cc:1397] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.171s user 0.013s sys 0.019s
I20260923 17:31:37.106510 17146 raft_consensus.cc:348] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.107144 17146 raft_consensus.cc:374] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:37.107366 17146 raft_consensus.cc:729] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:37.107939 17146 consensus_queue.cc:260] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.111136 17146 ts_tablet_manager.cc:1428] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260923 17:31:37.113286 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet e329dd46a3a245b6826b01ac8de56621 (table default.Table_5626751730cc42bebea406b124700924 [id=61954d5e2506486796d76b8d0d1f6d3f]). Expected version 1 got 0
I20260923 17:31:37.114420 17146 tablet_bootstrap.cc:492] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:37.114789 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet bd53fd73fce64b658b09a0fbfcdbb0fd (table default.Table_c1bfc0841ebe4015841fdddeeae9b6e9 [id=8abd6be5200d427c80760d3772ccabce]). Expected version 1 got 0
I20260923 17:31:37.116488 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
I20260923 17:31:37.117961 17143 ts_tablet_manager.cc:1428] T 4cd483bcad624c9a86503bdf15b9d6f8 P c36e79694e154f84a46980719217d7f2: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260923 17:31:37.120682 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 354f66c852b94b1d9f44b6bc012cf81b (table default.Table_f1ae923eae9c4a5fa1cd2c818fc374b9 [id=b6361ecc549d4f53a624d46842a9444e]). Expected version 1 got 0
I20260923 17:31:37.122210 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet b77ef164608f4ced8b1c2ecab6f88c1a (table default.Table_b0ce21ff6a6e46c8a4a37cdc3ab4f877 [id=a67f05c6ae754dcba81905a4dd293a0f]). Expected version 3 got 0
I20260923 17:31:37.122697 16379 catalog_manager.cc:5587] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559) does not have the latest schema for tablet 070d38df3f0f43edab3826835c9503ce (table default.Table_04688becafca4883b1c56cbe6ee8132d [id=ef4aea9a69ae43e48cf913c554fcdac9]). Expected version 1 got 0
I20260923 17:31:37.126984 17146 tablet_bootstrap.cc:654] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:37.130560 16862 tablet_service.cc:1514] Processing DeleteTablet for tablet 261d1d03290546969a74ca60beb890eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:19 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:37.132652 16862 tablet_service.cc:1514] Processing DeleteTablet for tablet c48bda995b284cf6be7bb6f348ec0fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:31 UTC) from {username='slave'} at 127.0.0.1:36510
I20260923 17:31:37.133209 17143 tablet_bootstrap.cc:492] T 354f66c852b94b1d9f44b6bc012cf81b P c36e79694e154f84a46980719217d7f2: Bootstrap starting.
I20260923 17:31:37.168919 18219 ts_tablet_manager.cc:1918] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:37.169281 18219 log.cc:1196] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/4cd483bcad624c9a86503bdf15b9d6f8
I20260923 17:31:37.169921 18219 ts_tablet_manager.cc:1939] T 4cd483bcad624c9a86503bdf15b9d6f8 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:37.171846 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 258) took 2315 ms. Trace:
I20260923 17:31:37.182615 17146 tablet_bootstrap.cc:492] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:37.183048 17146 ts_tablet_manager.cc:1397] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.069s user 0.008s sys 0.000s
I20260923 17:31:37.185454 17146 raft_consensus.cc:348] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.186074 17146 raft_consensus.cc:374] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:37.186295 17146 raft_consensus.cc:729] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:37.186856 17146 consensus_queue.cc:260] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.195802 17146 ts_tablet_manager.cc:1428] T e17107ce0b3c4a739b60a2667a50963f P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260923 17:31:37.196501 18221 ts_tablet_manager.cc:1918] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 17:31:37.196699 17146 tablet_bootstrap.cc:492] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:37.196904 18221 log.cc:1196] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/456273cf57314cb8a05c8e765543913f
I20260923 17:31:37.197641 18221 ts_tablet_manager.cc:1939] T 456273cf57314cb8a05c8e765543913f P c36e79694e154f84a46980719217d7f2: Deleting consensus metadata
I20260923 17:31:37.172137 18219 rpcz_store.cc:276] 0923 17:31:34.856609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.856733 (+ 124us) service_pool.cc:227] Handling call
0923 17:31:37.086656 (+2229923us) ts_tablet_manager.cc:1145] Deleting tablet 4cd483bcad624c9a86503bdf15b9d6f8
0923 17:31:37.086702 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.154904 (+ 68202us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.168700 (+ 13796us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.171777 (+ 3077us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2229066}
I20260923 17:31:37.206315 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 260) took 2318 ms. Trace:
I20260923 17:31:37.206449 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 4cd483bcad624c9a86503bdf15b9d6f8 (table default.Table_67ae78aa612d4ebcbaf1dcb60c98be71 [id=bbdc46a7e84c45198fc559139de7c792]) successfully deleted
I20260923 17:31:37.206636 18416 raft_consensus.cc:482] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:37.206615 18219 rpcz_store.cc:276] 0923 17:31:34.887781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.887877 (+ 96us) service_pool.cc:227] Handling call
0923 17:31:37.205995 (+2318118us) ts_tablet_manager.cc:1145] Deleting tablet c48bda995b284cf6be7bb6f348ec0fd0
0923 17:31:37.206052 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.206257 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2317495}
I20260923 17:31:37.207204 18416 raft_consensus.cc:504] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
W20260923 17:31:37.207780 16373 catalog_manager.cc:4808] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): delete failed for tablet c48bda995b284cf6be7bb6f348ec0fd0 because the tablet was not found. No further retry: Not found: Tablet not found: c48bda995b284cf6be7bb6f348ec0fd0
I20260923 17:31:37.208930 16379 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_80b04aa8dd0c4d07a921ae68656507c2 [id=725a68acaa3b4d16aed1d0fa620137c2] alter complete (version 1)
I20260923 17:31:37.209299 18416 leader_election.cc:288] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
I20260923 17:31:37.210991 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54f76e45d254086983cf6ee21484063" candidate_uuid: "e5c071cdf65042a59cad226a15a5532c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36e79694e154f84a46980719217d7f2" is_pre_election: true
W20260923 17:31:37.212280 16972 leader_election.cc:341] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:31:37.214128 18219 tablet_replica.cc:330] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:37.217077 18219 raft_consensus.cc:2229] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:37.217386 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 239) took 5012 ms. Trace:
I20260923 17:31:37.217756 18221 rpcz_store.cc:276] 0923 17:31:32.204589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.208706 (+ 4117us) service_pool.cc:227] Handling call
0923 17:31:36.971991 (+4763285us) ts_tablet_manager.cc:1145] Deleting tablet 456273cf57314cb8a05c8e765543913f
0923 17:31:36.972546 (+ 555us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.102720 (+130174us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.196198 (+ 93478us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.217317 (+ 21119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4762713}
I20260923 17:31:37.219152 16372 catalog_manager.cc:4841] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): tablet 456273cf57314cb8a05c8e765543913f (table default.Table_bf7ef441cc5e439092b7d3133a346bc7 [id=deee1461a77d4263ba3fb9bc01d1f18e]) successfully deleted
I20260923 17:31:37.223521 18221 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36510 (request call id 241) took 4832 ms. Trace:
I20260923 17:31:37.224159 18219 raft_consensus.cc:2258] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:37.224936 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet a84e322a4de24cbabf450fceb098d6e7 (table default.Table_0d6d9792c86e4eb7b9dbfaa0b3fa0d95 [id=36caf250f5024edba1821ab5af3d3525]). Expected version 1 got 0
I20260923 17:31:37.226125 18219 ts_tablet_manager.cc:1905] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:37.221555 17146 tablet_bootstrap.cc:654] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:37.230847 16975 connection.cc:535] server connection from 127.15.246.129:36239 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.232065 16972 connection.cc:535] client connection to 127.15.246.129:38879 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.232504 16972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:31:37.226476 16384 catalog_manager.cc:5587] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433) does not have the latest schema for tablet 523963a1d7fb4ac0b4c9fb3ddde7fe6c (table default.Table_4bd2aa1e022c48b7841a7aae1ba594a4 [id=2eaef63c05834c3ca1a9da799a6cbd2a]). Expected version 3 got 0
W20260923 17:31:37.231580 16828 connection.cc:535] server connection from 127.15.246.129:46475 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:31:37.214114 17143 tablet_bootstrap.cc:654] T 354f66c852b94b1d9f44b6bc012cf81b P c36e79694e154f84a46980719217d7f2: Neither blocks nor log segments found. Creating new log.
W20260923 17:31:37.238750 16374 connection.cc:535] server connection from 127.15.246.129:58511 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.239418 16372 connection.cc:535] client connection to 127.15.246.129:38879 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.239756 16372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 5 similar messages]
I20260923 17:31:37.238595 18221 rpcz_store.cc:276] 0923 17:31:32.391063 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:32.391600 (+ 537us) service_pool.cc:227] Handling call
0923 17:31:37.223092 (+4831492us) ts_tablet_manager.cc:1145] Deleting tablet 1feebaa76bcd4ff5b707814191cb3341
0923 17:31:37.223148 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.223463 (+ 315us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4662424}
I20260923 17:31:37.245329 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cd483bcad624c9a86503bdf15b9d6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:29 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:37.248924 16826 connection.cc:535] client connection to 127.15.246.129:38879 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.249347 16826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:31:37.260774 18221 tablet_replica.cc:330] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: stopping tablet replica
I20260923 17:31:37.261735 18221 raft_consensus.cc:2229] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:37.261762 18219 ts_tablet_manager.cc:1918] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:31:37.262115 18219 log.cc:1196] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/6e20fc94eecf4a14840ca88177070e33
I20260923 17:31:37.262250 18221 pending_rounds.cc:62] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Trying to abort 1 pending ops.
I20260923 17:31:37.262722 18219 ts_tablet_manager.cc:1939] T 6e20fc94eecf4a14840ca88177070e33 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
I20260923 17:31:37.262446 18221 pending_rounds.cc:68] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7332596502388998144 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "db3504813e9c4698a546b6d6fa0fbcb6" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_206108d7a4384c5e8a3201f96c3cb705" dest_uuid: "be93fda0843b44f5abc49e6f35c69217" new_extra_config { } }
I20260923 17:31:37.264050 18221 raft_consensus.cc:2258] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:37.271956 18221 ts_tablet_manager.cc:1905] T db3504813e9c4698a546b6d6fa0fbcb6 P c36e79694e154f84a46980719217d7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:31:37.272917 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet db3504813e9c4698a546b6d6fa0fbcb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 17:31:32 UTC) from {username='slave'} at 127.0.0.1:38390
W20260923 17:31:37.276027 16972 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.246.129:38879, user_credentials={real_user=slave}} blocked reactor thread for 43566.2us
I20260923 17:31:37.278308 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16799
I20260923 17:31:37.309381 17146 tablet_bootstrap.cc:492] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c: No bootstrap required, opened a new log
I20260923 17:31:37.309979 17146 ts_tablet_manager.cc:1397] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c: Time spent bootstrapping tablet: real 0.113s user 0.006s sys 0.005s
I20260923 17:31:37.312966 16384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 1dbd0d3324414f26ab5d34c375921ef2: default.Table_c1bfc0841ebe4015841fdddeeae9b6e9 [id=8abd6be5200d427c80760d3772ccabce] alter complete (version 1)
I20260923 17:31:37.314762 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 261) took 2353 ms. Trace:
I20260923 17:31:37.315503 18219 rpcz_store.cc:276] 0923 17:31:34.961386 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:34.961490 (+ 104us) service_pool.cc:227] Handling call
0923 17:31:37.213113 (+2251623us) ts_tablet_manager.cc:1145] Deleting tablet 6e20fc94eecf4a14840ca88177070e33
0923 17:31:37.213971 (+ 858us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.251469 (+ 37498us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.261556 (+ 10087us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.314696 (+ 53140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2251092}
I20260923 17:31:37.316848 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 6e20fc94eecf4a14840ca88177070e33 (table default.Table_005aa780a0d3449e928afd88676b3c66 [id=0fee7193bff9481d8360735a984d8715]) successfully deleted
I20260923 17:31:37.317762 18219 tablet_replica.cc:330] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
W20260923 17:31:37.319878 16372 catalog_manager.cc:4821] TS c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): delete failed for tablet 1feebaa76bcd4ff5b707814191cb3341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1feebaa76bcd4ff5b707814191cb3341 already in progress: creating tablet
W20260923 17:31:37.322216 16372 catalog_manager.cc:4584] TS be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 006895667f324d049debbebb132ed09b: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.312417 17146 raft_consensus.cc:348] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.326893 17146 raft_consensus.cc:374] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:37.327986 17146 raft_consensus.cc:729] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c071cdf65042a59cad226a15a5532c, State: Initialized, Role: FOLLOWER
I20260923 17:31:37.334923 18416 raft_consensus.cc:482] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:37.335458 18416 raft_consensus.cc:504] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.337527 18416 leader_election.cc:288] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433), be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879)
W20260923 17:31:37.338217 16972 leader_election.cc:334] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.338616 16972 leader_election.cc:302] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.339326 18416 raft_consensus.cc:2726] T 354f66c852b94b1d9f44b6bc012cf81b P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:37.340193 16972 consensus_peers.cc:487] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.340605 16972 consensus_peers.cc:487] T a2e5bc4ae186404c896814b95d6cc7ef P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.340932 16972 consensus_peers.cc:487] T 151184083ebb4ae4bf24fe171d33cfab P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.341261 16972 consensus_peers.cc:487] T bd53fd73fce64b658b09a0fbfcdbb0fd P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.341609 16972 consensus_peers.cc:487] T e92cfe501505458ea46da095a8c0d30a P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.341919 16972 leader_election.cc:334] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.335686 17146 consensus_queue.cc:260] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be93fda0843b44f5abc49e6f35c69217" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 38879 } } peers { permanent_uuid: "c36e79694e154f84a46980719217d7f2" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 35433 } } peers { permanent_uuid: "e5c071cdf65042a59cad226a15a5532c" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 42559 } }
I20260923 17:31:37.346319 17146 ts_tablet_manager.cc:1428] T d15e626ea47f435b8cc75d91d3d04900 P e5c071cdf65042a59cad226a15a5532c: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260923 17:31:37.342242 16972 leader_election.cc:302] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.349221 17146 tablet_bootstrap.cc:492] T 20382820ba9d490cab3299a29c378d2f P e5c071cdf65042a59cad226a15a5532c: Bootstrap starting.
I20260923 17:31:37.355072 17146 tablet_bootstrap.cc:654] T 20382820ba9d490cab3299a29c378d2f P e5c071cdf65042a59cad226a15a5532c: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:37.356423 18416 raft_consensus.cc:2726] T 80b11a6fd84840178b81d2b7610c7351 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:37.357437 16972 leader_election.cc:334] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.357815 16972 leader_election.cc:302] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.358438 18416 raft_consensus.cc:2726] T 11123f6ee880405ea76479c80fcea7e7 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:31:37.360637 16972 consensus_peers.cc:487] T 14bec348cce841a689f69bacdf21004d P e5c071cdf65042a59cad226a15a5532c -> Peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Couldn't send request to peer be93fda0843b44f5abc49e6f35c69217. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260923 17:31:37.361078 16972 leader_election.cc:334] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.361406 16972 leader_election.cc:302] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.363432 18416 raft_consensus.cc:2726] T 8a992ad5280444dda64c1325da92a707 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:37.364519 18219 raft_consensus.cc:2229] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 1 LEADER]: Raft consensus shutting down.
I20260923 17:31:37.365461 18219 raft_consensus.cc:2258] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:37.367270 18219 ts_tablet_manager.cc:1905] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:37.364496 16972 leader_election.cc:334] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.380646 16972 leader_election.cc:302] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.388424 18416 raft_consensus.cc:2726] T e54f76e45d254086983cf6ee21484063 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:37.394711 18219 ts_tablet_manager.cc:1918] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260923 17:31:37.395750 18219 log.cc:1196] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/191292e60d0d46fb89b52390feb48f80
I20260923 17:31:37.396410 18219 ts_tablet_manager.cc:1939] T 191292e60d0d46fb89b52390feb48f80 P e5c071cdf65042a59cad226a15a5532c: Deleting consensus metadata
W20260923 17:31:37.396447 16972 leader_election.cc:334] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer be93fda0843b44f5abc49e6f35c69217 (127.15.246.129:38879): Network error: Client connection negotiation failed: client connection to 127.15.246.129:38879: connect: Connection refused (error 111)
I20260923 17:31:37.398423 18219 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38390 (request call id 262) took 2120 ms. Trace:
I20260923 17:31:37.398715 18219 rpcz_store.cc:276] 0923 17:31:35.278244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:35.278377 (+ 133us) service_pool.cc:227] Handling call
0923 17:31:37.317580 (+2039203us) ts_tablet_manager.cc:1145] Deleting tablet 191292e60d0d46fb89b52390feb48f80
0923 17:31:37.317638 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0923 17:31:37.383148 (+ 65510us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.394522 (+ 11374us) tablet_metadata.cc:641] Metadata flushed
0923 17:31:37.398372 (+ 3850us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":45596,"tablet-delete.queue_time_us":2038484}
I20260923 17:31:37.399878 16373 catalog_manager.cc:4841] TS e5c071cdf65042a59cad226a15a5532c (127.15.246.131:42559): tablet 191292e60d0d46fb89b52390feb48f80 (table default.Table_0cb3da9761eb4c00b832368882e9e6cf [id=fb4487f3f20d4625a50510c06d8a8ec3]) successfully deleted
I20260923 17:31:37.413326 18219 tablet_replica.cc:330] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: stopping tablet replica
I20260923 17:31:37.417802 18219 raft_consensus.cc:2229] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:31:37.418387 18219 raft_consensus.cc:2258] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:31:37.420169 18219 ts_tablet_manager.cc:1905] T d2796fcb72f5497a92208b40224739e9 P e5c071cdf65042a59cad226a15a5532c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:31:37.423189 16374 connection.cc:535] server connection from 127.15.246.130:46845 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.423823 16972 connection.cc:535] client connection to 127.15.246.130:35433 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.423969 16453 connection.cc:535] server connection from 127.15.246.130:58279 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.428594 18426 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 17:31:37.425731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.246.130:35433 (local address 127.15.246.131:36947)
0923 17:31:37.426470 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0923 17:31:37.426544 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 17:31:37.427037 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 17:31:37.427595 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.246.130:35433: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":279}
W20260923 17:31:37.429159 16972 leader_election.cc:334] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c36e79694e154f84a46980719217d7f2 (127.15.246.130:35433): Network error: Client connection negotiation failed: client connection to 127.15.246.130:35433: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:31:37.429526 16972 leader_election.cc:302] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5c071cdf65042a59cad226a15a5532c; no voters: be93fda0843b44f5abc49e6f35c69217, c36e79694e154f84a46980719217d7f2
I20260923 17:31:37.430523 18416 raft_consensus.cc:2726] T 3d96d172fcb94589ae9cc36248176c17 P e5c071cdf65042a59cad226a15a5532c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:31:37.431974 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16948
W20260923 17:31:37.435916 16372 connection.cc:535] client connection to 127.15.246.130:35433 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:31:37.532330 16499 debug-util.cc:398] Leaking SignalData structure 0x7b080005c440 after lost signal to thread 16432
I20260923 17:31:37.532740 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16355
W20260923 17:31:37.533311 16499 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad320 after lost signal to thread 16502
I20260923 17:31:37.615721 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 16428
I20260923 17:31:37.656291 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 17244
2026-09-23T17:31:37Z chronyd exiting
I20260923 17:31:37.711776 16346 master-stress-test.cc:586] Tables created: 126
I20260923 17:31:37.711989 16346 master-stress-test.cc:587] Tables altered: 53
I20260923 17:31:37.712144 16346 master-stress-test.cc:588] Tables deleted: 43
I20260923 17:31:37.712323 16346 master-stress-test.cc:589] Tablets replaced: 20
I20260923 17:31:37.712484 16346 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (43494 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260923 17:31:37.793241 16346 test_util.cc:268] Using random seed: -620852543
2026-09-23T17:31:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
Could not open connection to daemon
2026-09-23T17:31:37Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMyBI3K/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMyBI3K/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/security.properties
2026-09-23 17:31:44: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMyBI3K/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskMyBI3K/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260923 17:31:52.682269 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.190:46265
--webserver_interface=127.15.246.190
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--hive_metastore_uris=thrift://127.0.0.1:38515
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260923 17:31:53.015475 18669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:53.016191 18669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:53.016714 18669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:53.049708 18669 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260923 17:31:53.050055 18669 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260923 17:31:53.050320 18669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:31:53.050568 18669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:53.050818 18669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:31:53.051056 18669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:31:53.091074 18669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38515
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.190:46265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.246.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:53.092671 18669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:53.094626 18669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:53.114128 18677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:53.114682 18679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:53.115779 18669 server_base.cc:1034] running on GCE node
W20260923 17:31:53.114181 18676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:54.229135 18669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:54.232522 18669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:54.233990 18669 hybrid_clock.cc:648] HybridClock initialized: now 1790184714233944 us; error 82 us; skew 500 ppm
I20260923 17:31:54.234869 18669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:54.246198 18669 webserver.cc:460] Webserver started at http://127.15.246.190:41557/ using document root <none> and password file <none>
I20260923 17:31:54.247355 18669 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:54.247562 18669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:54.248025 18669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:54.253650 18669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data/instance:
uuid: "ca93f1f37ccd46c08821589340bf34e1"
format_stamp: "Formatted at 2026-09-23 17:31:54 on dist-test-slave-mk2q"
I20260923 17:31:54.254963 18669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal/instance:
uuid: "ca93f1f37ccd46c08821589340bf34e1"
format_stamp: "Formatted at 2026-09-23 17:31:54 on dist-test-slave-mk2q"
I20260923 17:31:54.263381 18669 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260923 17:31:54.270064 18686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:54.271502 18669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.004s
I20260923 17:31:54.271886 18669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal
uuid: "ca93f1f37ccd46c08821589340bf34e1"
format_stamp: "Formatted at 2026-09-23 17:31:54 on dist-test-slave-mk2q"
I20260923 17:31:54.272286 18669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:54.325289 18669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:54.327159 18669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:54.327632 18669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:54.405514 18669 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.190:46265
I20260923 17:31:54.405610 18737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.190:46265 every 8 connection(s)
I20260923 17:31:54.408490 18669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/data/info.pb
I20260923 17:31:54.411278 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 18669
I20260923 17:31:54.411712 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-0/wal/instance
I20260923 17:31:54.413913 18738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:54.414517 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:35267
--webserver_interface=127.15.246.189
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--hive_metastore_uris=thrift://127.0.0.1:38515
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:31:54.425263 18738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:54.455847 18738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:31:54.458247 18690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:31:54.462750 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:31:54.512266 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 1
W20260923 17:31:54.515961 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:31:54.584604 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 2
W20260923 17:31:54.588806 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:31:54.680315 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 3
W20260923 17:31:54.684293 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:31:54.753714 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:54.754369 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:54.754810 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:54.789007 18739 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260923 17:31:54.789319 18739 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260923 17:31:54.789536 18739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:31:54.789733 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:54.789929 18739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:31:54.790136 18739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:31:54.827878 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 4
W20260923 17:31:54.831681 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:31:54.832927 18739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38515
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.246.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:54.834378 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:54.836200 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:54.856083 18750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:55.131222 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 5
W20260923 17:31:55.136251 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:31:55.683723 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 6
W20260923 17:31:55.691115 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.189:35267: Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:31:54.856196 18749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:54.858143 18739 server_base.cc:1034] running on GCE node
W20260923 17:31:54.856719 18752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:55.963562 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:55.966586 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:55.968087 18739 hybrid_clock.cc:648] HybridClock initialized: now 1790184715968028 us; error 59 us; skew 500 ppm
I20260923 17:31:55.969040 18739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:55.976413 18739 webserver.cc:460] Webserver started at http://127.15.246.189:36523/ using document root <none> and password file <none>
I20260923 17:31:55.977453 18739 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:55.977672 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:55.978188 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:55.983582 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/instance:
uuid: "05a13f8d28964563b4072f7542fd178b"
format_stamp: "Formatted at 2026-09-23 17:31:55 on dist-test-slave-mk2q"
I20260923 17:31:55.984848 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal/instance:
uuid: "05a13f8d28964563b4072f7542fd178b"
format_stamp: "Formatted at 2026-09-23 17:31:55 on dist-test-slave-mk2q"
I20260923 17:31:55.992902 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260923 17:31:55.999158 18761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:56.000425 18739 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 17:31:56.000830 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
uuid: "05a13f8d28964563b4072f7542fd178b"
format_stamp: "Formatted at 2026-09-23 17:31:55 on dist-test-slave-mk2q"
I20260923 17:31:56.001200 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:56.041307 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:56.042972 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:56.043440 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:56.122624 18739 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.189:35267
I20260923 17:31:56.122720 18813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.189:35267 every 8 connection(s)
I20260923 17:31:56.125674 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
I20260923 17:31:56.127990 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 18739
I20260923 17:31:56.128646 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal/instance
I20260923 17:31:56.131263 18814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:56.131848 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:41441
--webserver_interface=127.15.246.188
--webserver_port=0
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--hive_metastore_uris=thrift://127.0.0.1:38515
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:31:56.142794 18814 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:56.166152 18814 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:56.183935 18814 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:31:56.186674 18762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:31:56.191094 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.240773 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 1
W20260923 17:31:56.245167 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.313609 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 2
W20260923 17:31:56.317731 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.409157 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 3
W20260923 17:31:56.413611 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:31:56.462733 18815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:56.463371 18815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:56.463801 18815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:31:56.496614 18815 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260923 17:31:56.496941 18815 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260923 17:31:56.497242 18815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:31:56.497562 18815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:31:56.497862 18815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:31:56.498119 18815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:31:56.536928 18815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38515
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:41441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.246.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:56.538444 18815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:56.540465 18815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 17:31:56.557108 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 4
W20260923 17:31:56.561419 18825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:31:56.562008 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.751565 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } attempt: 7
I20260923 17:31:56.765070 18738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } has no permanent_uuid. Determining permanent_uuid...
W20260923 17:31:56.770828 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.829244 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 1
W20260923 17:31:56.833695 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.861435 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 5
W20260923 17:31:56.864786 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:56.915081 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 2
W20260923 17:31:56.919636 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:57.005067 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 3
W20260923 17:31:57.009465 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:57.149878 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 4
W20260923 17:31:57.154521 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:57.412454 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 6
W20260923 17:31:57.419041 18814 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:31:57.437877 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 5
W20260923 17:31:57.441987 18738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.246.188:41441: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:31:56.561841 18828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:56.564689 18815 server_base.cc:1034] running on GCE node
W20260923 17:31:56.562055 18826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:57.678970 18815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:31:57.681674 18815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:31:57.683107 18815 hybrid_clock.cc:648] HybridClock initialized: now 1790184717683069 us; error 62 us; skew 500 ppm
I20260923 17:31:57.683956 18815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:31:57.690191 18815 webserver.cc:460] Webserver started at http://127.15.246.188:41541/ using document root <none> and password file <none>
I20260923 17:31:57.691147 18815 fs_manager.cc:359] Metadata directory not provided
I20260923 17:31:57.691346 18815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:31:57.691818 18815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:31:57.697136 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/instance:
uuid: "469a88f1fddf4ecba558f342a2fe37fd"
format_stamp: "Formatted at 2026-09-23 17:31:57 on dist-test-slave-mk2q"
I20260923 17:31:57.698320 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal/instance:
uuid: "469a88f1fddf4ecba558f342a2fe37fd"
format_stamp: "Formatted at 2026-09-23 17:31:57 on dist-test-slave-mk2q"
I20260923 17:31:57.705732 18815 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260923 17:31:57.711495 18840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:31:57.712730 18815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 17:31:57.713078 18815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
uuid: "469a88f1fddf4ecba558f342a2fe37fd"
format_stamp: "Formatted at 2026-09-23 17:31:57 on dist-test-slave-mk2q"
I20260923 17:31:57.713392 18815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:31:57.742990 18815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:31:57.744500 18815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:31:57.744899 18815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:31:57.815832 18815 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.188:41441
I20260923 17:31:57.815994 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.188:41441 every 8 connection(s)
I20260923 17:31:57.818615 18815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
I20260923 17:31:57.823797 18892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:31:57.825978 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 18815
I20260923 17:31:57.826435 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal/instance
I20260923 17:31:57.838218 18892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:57.871932 18892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:57.884325 18892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } has no permanent_uuid. Determining permanent_uuid...
I20260923 17:31:57.903258 18892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Bootstrap starting.
I20260923 17:31:57.910871 18892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:57.913256 18892 log.cc:824] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:57.919525 18892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: No bootstrap required, opened a new log
I20260923 17:31:57.942046 18892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:57.942946 18892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:57.943317 18892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469a88f1fddf4ecba558f342a2fe37fd, State: Initialized, Role: FOLLOWER
I20260923 17:31:57.944444 18892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:57.947376 18900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:57.948074 18900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:57.949005 18892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: configured and running, proceeding with master startup.
W20260923 17:31:57.985178 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:57.985790 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:57.995687 18738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 6
W20260923 17:31:58.000154 18846 tablet.cc:2369] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 17:31:58.009805 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.013091 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:57.978274 18815 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: OK, instance_id { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" instance_seqno: 1790184714383760 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 17:31:58.017066 18815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:31:58.027345 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.027801 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.042289 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 17:31:58.045648 18738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Bootstrap starting.
W20260923 17:31:58.045761 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.053588 18738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:58.056113 18738 log.cc:824] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Log is configured to *not* fsync() on all Append() calls
W20260923 17:31:58.063809 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.065322 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.066347 18738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: No bootstrap required, opened a new log
W20260923 17:31:58.082687 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.083145 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.096547 18738 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.097463 18738 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:58.097736 18738 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca93f1f37ccd46c08821589340bf34e1, State: Initialized, Role: FOLLOWER
W20260923 17:31:58.098011 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.098380 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.098551 18738 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.101877 18919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.102882 18919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:58.103417 18738 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 17:31:58.112922 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.113546 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.128150 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.128592 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.144495 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.144982 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.159786 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.160351 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.175009 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.175383 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.181396 18900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:31:58.181831 18900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.187155 18713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:31:58.188010 18713 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 469a88f1fddf4ecba558f342a2fe37fd in term 0.
I20260923 17:31:58.184188 18900 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
I20260923 17:31:58.189358 18842 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1; no voters:
I20260923 17:31:58.197613 18900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:31:58.197949 18900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:31:58.198237 18900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:58.189968 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.204883 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.206171 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.206636 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.209972 18900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.214599 18713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1"
I20260923 17:31:58.214901 18900 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 1 election: Requested vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
I20260923 17:31:58.215247 18713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:31:58.222141 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.225719 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.230947 18713 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 469a88f1fddf4ecba558f342a2fe37fd in term 1.
I20260923 17:31:58.234967 18842 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1; no voters:
W20260923 17:31:58.243875 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.244719 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.245057 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.244201 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.245479 18900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:31:58.251046 18900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 LEADER]: Becoming Leader. State: Replica: 469a88f1fddf4ecba558f342a2fe37fd, State: Running, Role: LEADER
I20260923 17:31:58.253413 18900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
W20260923 17:31:58.259768 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.260183 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.192265 18669 master_runner.cc:186] Error getting master registration for 127.15.246.189:35267: OK, instance_id { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" instance_seqno: 1790184716100794 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 17:31:58.270180 18669 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:31:58.274822 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.275306 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.284507 18936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a88f1fddf4ecba558f342a2fe37fd. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
W20260923 17:31:58.286595 18914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 17:31:58.286616 18936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: LEADER
W20260923 17:31:58.286972 18914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.289878 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.290314 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.296830 18938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:31:58.303972 18938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260923 17:31:58.304566 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.304979 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.324342 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.325066 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.329931 18713 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Refusing update from remote peer 469a88f1fddf4ecba558f342a2fe37fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 17:31:58.331750 18936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:58.337332 18692 tablet.cc:2369] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:31:58.359783 18919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a88f1fddf4ecba558f342a2fe37fd. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.361570 18919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 17:31:58.366302 18934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 17:31:58.369707 18934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.371927 18939 mvcc.cc:205] Tried to move back new op lower bound from 7332596606260076544 to 7332596606028308480. Current Snapshot: MvccSnapshot[applied={T|T < 7332596606260076544}]
I20260923 17:31:58.374251 18900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.374990 18900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: LEADER
I20260923 17:31:58.377610 18940 mvcc.cc:205] Tried to move back new op lower bound from 7332596606260076544 to 7332596606028308480. Current Snapshot: MvccSnapshot[applied={T|T < 7332596606260076544}]
I20260923 17:31:58.387861 18938 catalog_manager.cc:1347] Generated new cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:31:58.388175 18938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:31:58.393502 18919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.394232 18919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:58.399947 18934 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:31:58.400887 18934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: loading cluster ID for follower catalog manager: success
W20260923 17:31:58.407063 18934 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 17:31:58.407465 18934 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260923 17:31:58.441872 18934 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: acquiring CA information for follower catalog manager: Not found: root CA entry not found
I20260923 17:31:58.446171 18938 catalog_manager.cc:1370] Generated new certificate authority record
W20260923 17:31:58.445835 18934 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.448944 18938 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:31:58.482838 18814 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } attempt: 7
I20260923 17:31:58.490599 18934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: acquiring CA information for follower catalog manager: success
I20260923 17:31:58.500913 18938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Generated new TSK 0
I20260923 17:31:58.502856 18938 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 17:31:58.508204 18934 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260923 17:31:58.510892 18938 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260923 17:31:58.515092 18938 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260923 17:31:58.532437 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.129:0
--local_ip_for_outbound_sockets=127.15.246.129
--webserver_interface=127.15.246.129
--webserver_port=0
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:31:58.573730 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Bootstrap starting.
I20260923 17:31:58.579361 18934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:31:58.589401 18814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Neither blocks nor log segments found. Creating new log.
I20260923 17:31:58.594754 18814 log.cc:824] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Log is configured to *not* fsync() on all Append() calls
I20260923 17:31:58.616281 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: No bootstrap required, opened a new log
I20260923 17:31:58.695713 18814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.698268 18814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:31:58.699054 18814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a13f8d28964563b4072f7542fd178b, State: Initialized, Role: FOLLOWER
I20260923 17:31:58.701323 18814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:58.723961 18787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b"
I20260923 17:31:58.725090 18788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
W20260923 17:31:58.726006 18788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42434 (request call id 1) took 541 ms (client timeout 718 ms). Trace:
W20260923 17:31:58.726336 18788 rpcz_store.cc:269] 0923 17:31:58.184963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:58.185165 (+ 202us) service_pool.cc:227] Handling call
0923 17:31:58.725930 (+540765us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260923 17:31:58.729740 18787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:31:58.737891 18787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 469a88f1fddf4ecba558f342a2fe37fd in term 1.
I20260923 17:31:58.740715 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.741681 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:58.729349 18814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:31:58.748608 18786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Refusing update from remote peer 469a88f1fddf4ecba558f342a2fe37fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260923 17:31:58.751863 18787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42434 (request call id 2) took 537 ms (client timeout 693 ms). Trace:
I20260923 17:31:58.751792 18900 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:31:58.752264 18787 rpcz_store.cc:269] 0923 17:31:58.214659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0923 17:31:58.214879 (+ 220us) service_pool.cc:227] Handling call
0923 17:31:58.747812 (+532933us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b54000118d4. stack: 00007f5e7146e033 00007f5e7146df30 00007f5e70fef534 00000000004db2b4 00000000004db249 00007f5e7837a471 00007f5e783f92ae 00007f5e793d1c27 00007f5e77ba9c04 00007f5e77ba9b92 00007f5e77ba9ac1 00007f5e77ba9a3c 00007f5e77ba8cf2 00007f5e761a494c 00007f5e761a3d86 00007f5e761a372f
0923 17:31:58.751807 (+ 3995us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11546496}
I20260923 17:31:58.825704 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a88f1fddf4ecba558f342a2fe37fd. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.826445 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:58.827534 18952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:58.828781 18952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:31:58.924492 18967 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 17:31:58.955377 18967 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:31:58.958195 18967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: loading cluster ID for follower catalog manager: success
I20260923 17:31:58.975033 18967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: acquiring CA information for follower catalog manager: success
I20260923 17:31:58.987686 18967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:31:59.015405 18969 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:31:59.017887 18969 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:59.032682 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:31:59.034032 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:31:59.037230 18690 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:31:59.040584 18969 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:31:59.041630 18969 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:59.221680 18945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:31:59.222344 18945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:31:59.223238 18945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 17:31:59.257354 18952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:31:59.257887 18952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:59.261658 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:31:59.263856 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:31:59.266371 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:31:59.267202 18947 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:31:59.260102 18952 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:31:59.290932 18945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260923 17:31:59.291457 18900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:31:59.292344 18900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: LEADER
W20260923 17:31:59.292448 18945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.129
I20260923 17:31:59.365578 18945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.246.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.129
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:31:59.367177 18945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:31:59.369417 18945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:31:59.423888 18979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:59.570472 18985 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:31:59.571201 18985 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:31:59.575517 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:31:59.578132 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:31:59.574205 18985 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:31:59.580344 18690 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:31:59.581123 18985 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:00.137010 18986 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:00.137730 18986 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:00.139891 18986 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:00.141911 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:00.143383 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:00.146219 18690 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:00.147198 18986 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:00.247385 18987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:00.250447 18987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:00.256640 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:00.261058 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:00.262835 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:00.270771 18987 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:00.271996 18987 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:00.705387 18989 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:00.706097 18989 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:00.710732 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:00.713769 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:00.715466 18687 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:00.718210 18989 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:00.719995 18989 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:31:59.434046 18978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:31:59.433245 18945 server_base.cc:1034] running on GCE node
W20260923 17:31:59.432606 18982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:01.002938 18945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:32:01.012953 18945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:32:01.016654 18945 hybrid_clock.cc:648] HybridClock initialized: now 1790184721016546 us; error 106 us; skew 500 ppm
I20260923 17:32:01.017961 18945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:32:01.049618 18945 webserver.cc:460] Webserver started at http://127.15.246.129:39305/ using document root <none> and password file <none>
I20260923 17:32:01.050946 18945 fs_manager.cc:359] Metadata directory not provided
I20260923 17:32:01.051232 18945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:32:01.051833 18945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:32:01.061312 18945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data/instance:
uuid: "dedc4fa600d8426c9074e3c8916aeaee"
format_stamp: "Formatted at 2026-09-23 17:32:01 on dist-test-slave-mk2q"
I20260923 17:32:01.062866 18945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal/instance:
uuid: "dedc4fa600d8426c9074e3c8916aeaee"
format_stamp: "Formatted at 2026-09-23 17:32:01 on dist-test-slave-mk2q"
I20260923 17:32:01.082222 18945 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.012s sys 0.004s
I20260923 17:32:01.098861 18995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:01.100485 18945 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.010s sys 0.001s
I20260923 17:32:01.101743 18945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal
uuid: "dedc4fa600d8426c9074e3c8916aeaee"
format_stamp: "Formatted at 2026-09-23 17:32:01 on dist-test-slave-mk2q"
I20260923 17:32:01.102190 18945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:32:01.200665 19000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:01.201215 19000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:01.204089 19000 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:01.204010 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:01.205461 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:01.207391 18763 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:01.208491 19000 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:01.213647 18945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:32:01.215893 18945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:32:01.216552 18945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:32:01.220369 18945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:32:01.226821 18945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:32:01.227139 18945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:01.227465 18945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:32:01.227691 18945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:01.481242 18945 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.129:37787
I20260923 17:32:01.481355 19109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.129:37787 every 8 connection(s)
I20260923 17:32:01.484150 18945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/data/info.pb
I20260923 17:32:01.486596 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 18945
I20260923 17:32:01.487171 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal/instance
I20260923 17:32:01.504737 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.130:0
--local_ip_for_outbound_sockets=127.15.246.130
--webserver_interface=127.15.246.130
--webserver_port=0
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:32:01.584272 19110 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:01.584848 19110 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:01.586193 19110 heartbeater.cc:511] Master 127.15.246.188:41441 requested a full tablet report, sending...
I20260923 17:32:01.590140 18857 ts_manager.cc:194] Registered new tserver with Master: dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787)
I20260923 17:32:01.593566 19111 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:01.593973 19111 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:01.594250 18857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.129:56949
I20260923 17:32:01.594935 19111 heartbeater.cc:511] Master 127.15.246.189:35267 requested a full tablet report, sending...
I20260923 17:32:01.598770 18778 ts_manager.cc:194] Registered new tserver with Master: dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787)
I20260923 17:32:01.605366 19115 heartbeater.cc:347] Connected to a master server at 127.15.246.190:46265
I20260923 17:32:01.605830 19115 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:01.606629 19115 heartbeater.cc:511] Master 127.15.246.190:46265 requested a full tablet report, sending...
I20260923 17:32:01.609952 18702 ts_manager.cc:194] Registered new tserver with Master: dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787)
I20260923 17:32:01.739120 19121 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:01.739513 19121 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:01.741978 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:01.741936 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:01.743472 18687 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:01.744254 19121 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:01.744750 19121 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:32:01.978790 19114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:32:01.979290 19114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:32:01.979848 19114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:32:02.013015 19114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:32:02.013945 19114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.130
I20260923 17:32:02.052062 19114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.246.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.130
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:32:02.053401 19114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:32:02.055051 19114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:32:02.076172 19130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:02.103089 19110 heartbeater.cc:503] Master 127.15.246.188:41441 was elected leader, sending a full tablet report...
I20260923 17:32:02.685325 19135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:02.685792 19135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:02.687841 19135 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:02.689397 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:02.689687 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:02.691625 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:02.692482 19135 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:02.787590 19136 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:02.788034 19136 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:02.789794 19136 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:02.790819 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:02.791950 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:02.793720 18687 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:02.794688 19136 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:32:02.076299 19132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:32:02.076176 19129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:02.077730 19114 server_base.cc:1034] running on GCE node
I20260923 17:32:03.226614 19114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:32:03.229266 19114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:32:03.230743 19114 hybrid_clock.cc:648] HybridClock initialized: now 1790184723230648 us; error 84 us; skew 500 ppm
I20260923 17:32:03.231746 19114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:32:03.238857 19114 webserver.cc:460] Webserver started at http://127.15.246.130:43585/ using document root <none> and password file <none>
I20260923 17:32:03.239912 19114 fs_manager.cc:359] Metadata directory not provided
I20260923 17:32:03.240145 19114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:32:03.240666 19114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:32:03.246677 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data/instance:
uuid: "2a823629310d45f880d1c553255a6d1b"
format_stamp: "Formatted at 2026-09-23 17:32:03 on dist-test-slave-mk2q"
I20260923 17:32:03.247998 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/instance:
uuid: "2a823629310d45f880d1c553255a6d1b"
format_stamp: "Formatted at 2026-09-23 17:32:03 on dist-test-slave-mk2q"
I20260923 17:32:03.256852 19114 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260923 17:32:03.264179 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:03.265532 19114 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260923 17:32:03.265964 19114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal
uuid: "2a823629310d45f880d1c553255a6d1b"
format_stamp: "Formatted at 2026-09-23 17:32:03 on dist-test-slave-mk2q"
I20260923 17:32:03.266333 19114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:32:03.305862 19114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:32:03.307548 19114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:32:03.308060 19114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:32:03.310920 19114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:32:03.315830 19114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:32:03.316125 19114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:03.316452 19114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:32:03.316607 19114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:03.359336 19173 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:03.360003 19173 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:03.363119 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:03.364599 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:03.363965 19173 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:03.366334 18687 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca93f1f37ccd46c08821589340bf34e1; no voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:03.367267 19173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:03.487275 19114 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.130:43119
I20260923 17:32:03.487396 19255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.130:43119 every 8 connection(s)
I20260923 17:32:03.490381 19114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/data/info.pb
I20260923 17:32:03.499784 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 19114
I20260923 17:32:03.500449 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/instance
I20260923 17:32:03.531143 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.246.131:0
--local_ip_for_outbound_sockets=127.15.246.131
--webserver_interface=127.15.246.131
--webserver_port=0
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260923 17:32:03.552013 19259 heartbeater.cc:347] Connected to a master server at 127.15.246.190:46265
I20260923 17:32:03.552600 19258 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:03.552727 19259 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:03.552973 19258 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:03.554639 19259 heartbeater.cc:511] Master 127.15.246.190:46265 requested a full tablet report, sending...
I20260923 17:32:03.555162 19258 heartbeater.cc:511] Master 127.15.246.189:35267 requested a full tablet report, sending...
I20260923 17:32:03.556308 19256 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:03.556684 19256 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:03.557587 19256 heartbeater.cc:511] Master 127.15.246.188:41441 requested a full tablet report, sending...
I20260923 17:32:03.558243 18778 ts_manager.cc:194] Registered new tserver with Master: 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:03.559932 18857 ts_manager.cc:194] Registered new tserver with Master: 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:03.561159 18702 ts_manager.cc:194] Registered new tserver with Master: 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:03.561797 18857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.130:35259
W20260923 17:32:03.860599 19267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:32:03.861148 19267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:32:03.861761 19267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:32:03.896610 19267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:32:03.897944 19267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.246.131
I20260923 17:32:03.934687 19267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.246.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.246.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.246.131
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:32:03.935913 19267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:32:03.937623 19267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:32:03.959353 19274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:04.048177 19279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:04.048667 19279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:04.051219 19279 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:04.051753 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:04.053223 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:04.055761 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:04.057612 19279 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:04.065796 19256 heartbeater.cc:503] Master 127.15.246.188:41441 was elected leader, sending a full tablet report...
I20260923 17:32:04.569926 19280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:04.570525 19280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:04.573053 19280 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:04.574605 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:04.574638 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:04.576313 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:04.577349 19280 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260923 17:32:03.959594 19273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:32:03.960353 19276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:03.962584 19267 server_base.cc:1034] running on GCE node
I20260923 17:32:05.110077 19267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:32:05.112722 19267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:32:05.114171 19267 hybrid_clock.cc:648] HybridClock initialized: now 1790184725114113 us; error 71 us; skew 500 ppm
I20260923 17:32:05.115195 19267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:32:05.123138 19267 webserver.cc:460] Webserver started at http://127.15.246.131:38093/ using document root <none> and password file <none>
I20260923 17:32:05.124439 19267 fs_manager.cc:359] Metadata directory not provided
I20260923 17:32:05.124691 19267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:32:05.125198 19267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 17:32:05.131273 19286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:05.132295 19286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:05.133507 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data/instance:
uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
format_stamp: "Formatted at 2026-09-23 17:32:05 on dist-test-slave-mk2q"
I20260923 17:32:05.135499 19286 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:05.135852 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:05.136711 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal/instance:
uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
format_stamp: "Formatted at 2026-09-23 17:32:05 on dist-test-slave-mk2q"
I20260923 17:32:05.139344 18867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:05.141213 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:05.142266 19286 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:05.147449 19267 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.000s
I20260923 17:32:05.154387 19288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:05.156040 19267 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260923 17:32:05.156502 19267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal
uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
format_stamp: "Formatted at 2026-09-23 17:32:05 on dist-test-slave-mk2q"
I20260923 17:32:05.156869 19267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:32:05.204922 19267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:32:05.206687 19267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:32:05.207182 19267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:32:05.210098 19267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 17:32:05.214811 19267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 17:32:05.215071 19267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:05.215365 19267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 17:32:05.215507 19267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:05.363313 19267 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.131:34485
I20260923 17:32:05.363471 19401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.131:34485 every 8 connection(s)
I20260923 17:32:05.366012 19267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/data/info.pb
I20260923 17:32:05.374301 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 19267
I20260923 17:32:05.374955 16346 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal/instance
I20260923 17:32:05.412871 19402 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:05.413453 19402 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:05.414356 19404 heartbeater.cc:347] Connected to a master server at 127.15.246.190:46265
I20260923 17:32:05.414762 19404 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:05.414865 19402 heartbeater.cc:511] Master 127.15.246.188:41441 requested a full tablet report, sending...
I20260923 17:32:05.415716 19404 heartbeater.cc:511] Master 127.15.246.190:46265 requested a full tablet report, sending...
I20260923 17:32:05.417861 18856 ts_manager.cc:194] Registered new tserver with Master: 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:05.418759 18702 ts_manager.cc:194] Registered new tserver with Master: 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:05.419988 18856 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.246.131:58797
I20260923 17:32:05.427192 19403 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:05.427604 19403 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:05.428402 19403 heartbeater.cc:511] Master 127.15.246.189:35267 requested a full tablet report, sending...
I20260923 17:32:05.430933 18778 ts_manager.cc:194] Registered new tserver with Master: 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:05.442304 16346 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 17:32:05.504390 18856 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53222:
name: "default.Table_319978fc115b4d10b8f8f3e8fac76165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.510421 18857 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53222:
name: "default.Table_e6ca980ce29c4245bb88eaf5eb38907b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.511893 18855 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53222:
name: "default.Table_1da9836e55d14639a899e7d3f4d58b55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.513422 18854 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53222:
name: "default.Table_78b2f53459124b318a200b9c57216ec9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:32:05.520536 18856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_319978fc115b4d10b8f8f3e8fac76165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:32:05.537753 16346 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu with pid 18815
W20260923 17:32:05.580047 19412 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.246.188:41441)
W20260923 17:32:05.583222 19143 connection.cc:535] client connection to 127.15.246.188:41441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:05.584055 19256 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:05.588657 19418 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 17:32:05.581909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.246.188:41441 (local address 127.0.0.1:53232)
0923 17:32:05.582553 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0923 17:32:05.582595 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 17:32:05.582912 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 17:32:05.587009 (+ 4097us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":116}
W20260923 17:32:05.642570 19110 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:05.674636 19437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:05.678638 19437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:05.682526 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca93f1f37ccd46c08821589340bf34e1" is_pre_election: true
I20260923 17:32:05.686913 19437 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:05.688946 18762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260923 17:32:05.694459 18762 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:05.699225 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:05.699823 19437 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 17:32:05.812412 19440 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:05.813409 19440 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:05.816358 19440 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:05.819756 18687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111) [suppressed 12 similar messages]
I20260923 17:32:05.825670 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:05.826491 18786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca93f1f37ccd46c08821589340bf34e1 in term 1.
I20260923 17:32:05.833176 18690 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1; no voters:
I20260923 17:32:05.834065 19440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 17:32:05.834542 19440 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:05.835000 19440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 1 FOLLOWER]: Advancing to term 2
W20260923 17:32:05.840682 18687 leader_election.cc:334] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:05.845924 19440 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:05.849812 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca93f1f37ccd46c08821589340bf34e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b"
I20260923 17:32:05.850538 18786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 1 FOLLOWER]: Advancing to term 2
W20260923 17:32:05.852639 18687 leader_election.cc:334] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:05.853293 19440 leader_election.cc:288] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 election: Requested vote from peers 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:05.863013 18786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca93f1f37ccd46c08821589340bf34e1 in term 2.
I20260923 17:32:05.867316 18690 leader_election.cc:302] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1; no voters: 469a88f1fddf4ecba558f342a2fe37fd
I20260923 17:32:05.868762 19440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 17:32:05.874511 19440 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [term 2 LEADER]: Becoming Leader. State: Replica: ca93f1f37ccd46c08821589340bf34e1, State: Running, Role: LEADER
I20260923 17:32:05.875497 19440 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:05.881296 19443 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca93f1f37ccd46c08821589340bf34e1. Latest consensus state: current_term: 2 leader_uuid: "ca93f1f37ccd46c08821589340bf34e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:05.882162 19443 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: This master's current role is: LEADER
I20260923 17:32:05.892930 19445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:32:05.902870 19445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 17:32:05.906510 19445 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:32:05.906735 19445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:32:05.914043 19445 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:32:05.917593 19445 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Loaded TSK: 0
I20260923 17:32:05.918963 19445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 17:32:05.921599 19445 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260923 17:32:05.925645 19445 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260923 17:32:05.925980 19402 heartbeater.cc:503] Master 127.15.246.188:41441 was elected leader, sending a full tablet report...
W20260923 17:32:05.929855 19402 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:05.947140 18703 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_1da9836e55d14639a899e7d3f4d58b55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.947513 18702 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_78b2f53459124b318a200b9c57216ec9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.949344 18700 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_319978fc115b4d10b8f8f3e8fac76165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:05.947531 18701 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_e6ca980ce29c4245bb88eaf5eb38907b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 17:32:05.961680 18702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_78b2f53459124b318a200b9c57216ec9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:32:05.965188 18703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1da9836e55d14639a899e7d3f4d58b55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:32:05.965517 18700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_319978fc115b4d10b8f8f3e8fac76165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260923 17:32:05.967919 18701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e6ca980ce29c4245bb88eaf5eb38907b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 17:32:06.153517 18702 catalog_manager.cc:2148] created HMS catalog entry for table [id=7d92bc7cedf94b75bfb904ad1c0468db]
W20260923 17:32:06.172462 18687 consensus_peers.cc:487] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 -> Peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Couldn't send request to peer 469a88f1fddf4ecba558f342a2fe37fd. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 17:32:06.174904 18786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Refusing update from remote peer ca93f1f37ccd46c08821589340bf34e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260923 17:32:06.176498 19443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260923 17:32:06.191596 19437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca93f1f37ccd46c08821589340bf34e1. Latest consensus state: current_term: 2 leader_uuid: "ca93f1f37ccd46c08821589340bf34e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:06.192487 19437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:06.207453 18701 catalog_manager.cc:2148] created HMS catalog entry for table [id=0d531f02e1834792b98bf7348fca884a]
I20260923 17:32:06.216531 19443 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ca93f1f37ccd46c08821589340bf34e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:06.217481 19443 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1 [sys.catalog]: This master's current role is: LEADER
I20260923 17:32:06.223428 18702 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33228:
table { table_name: "default.Table_78b2f53459124b318a200b9c57216ec9" } new_table_name: "default.Table_029ff3fda0d747e68a515ed3dd98d6e4" modify_external_catalogs: true
I20260923 17:32:06.225342 19437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ca93f1f37ccd46c08821589340bf34e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:06.226013 19437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:06.253715 18701 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33228:
table { table_name: "default.Table_e6ca980ce29c4245bb88eaf5eb38907b" } new_table_name: "default.Table_fd2cd848c438499980de5e4bae0b10b6" modify_external_catalogs: true
I20260923 17:32:06.341645 19045 tablet_service.cc:1467] Processing CreateTablet for tablet 6440b97f09304b988f67124faacaee58 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.347478 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6440b97f09304b988f67124faacaee58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.343036 19044 tablet_service.cc:1467] Processing CreateTablet for tablet e8a06927aa374ac280d04bb5df535d01 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.349128 19044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a06927aa374ac280d04bb5df535d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.356501 19190 tablet_service.cc:1467] Processing CreateTablet for tablet e8a06927aa374ac280d04bb5df535d01 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.359315 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a06927aa374ac280d04bb5df535d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.359915 19336 tablet_service.cc:1467] Processing CreateTablet for tablet 6440b97f09304b988f67124faacaee58 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.362789 19336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6440b97f09304b988f67124faacaee58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.365406 19335 tablet_service.cc:1467] Processing CreateTablet for tablet e8a06927aa374ac280d04bb5df535d01 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.357914 19191 tablet_service.cc:1467] Processing CreateTablet for tablet 6440b97f09304b988f67124faacaee58 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.366494 19335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a06927aa374ac280d04bb5df535d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.367241 19191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6440b97f09304b988f67124faacaee58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.405421 19460 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:06.411988 19460 tablet_bootstrap.cc:654] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.414209 19460 log.cc:824] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Log is configured to *not* fsync() on all Append() calls
I20260923 17:32:06.421556 19460 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:06.422166 19460 ts_tablet_manager.cc:1397] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.011s
I20260923 17:32:06.424413 18699 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_6a36064c5aba4b21bd8b5da2ccb2212e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:06.441291 18698 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_0ba166e019864d25946c15765c3f0b71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
17:32:06.443 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_319978fc115b4d10b8f8f3e8fac76165 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
W20260923 17:32:06.447965 18700 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=2daafd24e0e04fc1a872dd29bf5d1c98]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_319978fc115b4d10b8f8f3e8fac76165 already exists)
I20260923 17:32:06.454766 18700 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_af14483dad1b4c50a7033d85e9cbc651"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:06.456856 19460 raft_consensus.cc:348] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.457868 19460 raft_consensus.cc:374] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.458199 19460 raft_consensus.cc:729] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.459245 19460 consensus_queue.cc:260] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.481640 19464 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:06.484582 18697 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:06.495273 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:06.510588 18697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Sending DeleteTablet for 3 replicas of tablet 6440b97f09304b988f67124faacaee58
I20260923 17:32:06.513085 19336 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.520663 19259 heartbeater.cc:503] Master 127.15.246.190:46265 was elected leader, sending a full tablet report...
I20260923 17:32:06.522341 19191 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:38004
I20260923 17:32:06.517256 19460 ts_tablet_manager.cc:1428] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.095s user 0.049s sys 0.039s
I20260923 17:32:06.528805 18697 catalog_manager.cc:6565] ReplaceTablet: tablet 6440b97f09304b988f67124faacaee58 deleted and replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:06.528896 19465 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
W20260923 17:32:06.530746 19264 tablet.cc:2369] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:32:06.538306 19460 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:06.538879 19467 tablet_replica.cc:330] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: stopping tablet replica
I20260923 17:32:06.541770 19044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:37038
I20260923 17:32:06.548138 18703 catalog_manager.cc:2148] created HMS catalog entry for table [id=85c52abfb3044f52b73e4ce6e203b153]
I20260923 17:32:06.544152 19460 tablet_bootstrap.cc:654] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.555349 19467 raft_consensus.cc:2229] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:06.555969 19465 tablet_bootstrap.cc:654] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.563102 19467 raft_consensus.cc:2258] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.567306 19465 log.cc:824] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Log is configured to *not* fsync() on all Append() calls
I20260923 17:32:06.567452 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:41441
--webserver_interface=127.15.246.188
--webserver_port=41541
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--hive_metastore_uris=thrift://127.0.0.1:38515
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260923 17:32:06.580716 19045 tablet_service.cc:1467] Processing CreateTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.582015 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04c5431995d48c7b91f7bfad0ee90a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.587827 18703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33228:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
I20260923 17:32:06.588759 18703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33228:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
W20260923 17:32:06.593158 18690 catalog_manager.cc:4821] TS dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): delete failed for tablet 6440b97f09304b988f67124faacaee58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6440b97f09304b988f67124faacaee58 already in progress: creating tablet
F20260923 17:32:06.594501 18932 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_319978fc115b4d10b8f8f3e8fac76165
*** Check failure stack trace: ***
I20260923 17:32:06.597905 19191 tablet_service.cc:1467] Processing CreateTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.595923 19335 tablet_service.cc:1467] Processing CreateTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.599269 19191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04c5431995d48c7b91f7bfad0ee90a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.599972 19335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04c5431995d48c7b91f7bfad0ee90a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.605990 19465 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:06.606559 19465 ts_tablet_manager.cc:1397] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.088s user 0.008s sys 0.008s
I20260923 17:32:06.613145 19467 ts_tablet_manager.cc:1905] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:06.626703 19043 tablet_service.cc:1467] Processing CreateTablet for tablet 6753a0b86b7646339cec60f5c47811f1 (DEFAULT_TABLE table=default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.627996 19043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6753a0b86b7646339cec60f5c47811f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.627731 19334 tablet_service.cc:1467] Processing CreateTablet for tablet 6753a0b86b7646339cec60f5c47811f1 (DEFAULT_TABLE table=default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.629113 19334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6753a0b86b7646339cec60f5c47811f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.648151 19115 heartbeater.cc:503] Master 127.15.246.190:46265 was elected leader, sending a full tablet report...
I20260923 17:32:06.648692 19190 tablet_service.cc:1467] Processing CreateTablet for tablet 6753a0b86b7646339cec60f5c47811f1 (DEFAULT_TABLE table=default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.649739 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6753a0b86b7646339cec60f5c47811f1. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f04efc4c9a0 google::LogMessage::Flush() at ??:0
W20260923 17:32:06.670646 18688 catalog_manager.cc:4821] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): delete failed for tablet 6440b97f09304b988f67124faacaee58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6440b97f09304b988f67124faacaee58 already in progress: creating tablet
I20260923 17:32:06.676085 19467 ts_tablet_manager.cc:1918] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:06.676537 19467 log.cc:1196] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/6440b97f09304b988f67124faacaee58
I20260923 17:32:06.677348 19467 ts_tablet_manager.cc:1939] T 6440b97f09304b988f67124faacaee58 P 2a823629310d45f880d1c553255a6d1b: Deleting consensus metadata
I20260923 17:32:06.684535 19043 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:37038
W20260923 17:32:06.686435 18690 catalog_manager.cc:4821] TS dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): delete failed for tablet 6440b97f09304b988f67124faacaee58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6440b97f09304b988f67124faacaee58 already in progress: creating tablet
I20260923 17:32:06.688939 19464 tablet_bootstrap.cc:654] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.691387 19464 log.cc:824] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Log is configured to *not* fsync() on all Append() calls
I20260923 17:32:06.694056 18687 catalog_manager.cc:4841] TS 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): tablet 6440b97f09304b988f67124faacaee58 (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]) successfully deleted
I20260923 17:32:06.699095 19460 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:06.699525 19460 ts_tablet_manager.cc:1397] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.161s user 0.024s sys 0.034s
I20260923 17:32:06.701988 19460 raft_consensus.cc:348] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.702664 19460 raft_consensus.cc:374] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.702919 19460 raft_consensus.cc:729] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.703668 19460 consensus_queue.cc:260] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.706007 19460 ts_tablet_manager.cc:1428] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260923 17:32:06.712524 19460 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:06.723093 19460 tablet_bootstrap.cc:654] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.724408 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet b04c5431995d48c7b91f7bfad0ee90a6 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:06.725481 19464 tablet_bootstrap.cc:492] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:06.726037 19464 ts_tablet_manager.cc:1397] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.245s user 0.009s sys 0.007s
I20260923 17:32:06.735918 19460 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:06.736485 19460 ts_tablet_manager.cc:1397] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.009s
@ 0x7f04efc4c70f google::LogMessage::~LogMessage() at ??:0
I20260923 17:32:06.738216 18697 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:06.739259 19460 raft_consensus.cc:348] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.739957 19460 raft_consensus.cc:374] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.740195 19460 raft_consensus.cc:729] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.740832 19460 consensus_queue.cc:260] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.742794 19460 ts_tablet_manager.cc:1428] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260923 17:32:06.743685 19460 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:06.749203 19460 tablet_bootstrap.cc:654] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.749782 18696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Sending DeleteTablet for 3 replicas of tablet b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:06.749418 19465 raft_consensus.cc:348] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.750310 19465 raft_consensus.cc:374] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.750588 19465 raft_consensus.cc:729] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.751672 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 with delete_type TABLET_DATA_DELETED (replaced by tablet bb70d5dd96c44b9cac045d923f6d825c) from {username='slave'} at 127.0.0.1:37038
W20260923 17:32:06.753494 18690 catalog_manager.cc:4821] TS dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): delete failed for tablet b04c5431995d48c7b91f7bfad0ee90a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b04c5431995d48c7b91f7bfad0ee90a6 already in progress: creating tablet
I20260923 17:32:06.755324 18696 catalog_manager.cc:6565] ReplaceTablet: tablet b04c5431995d48c7b91f7bfad0ee90a6 deleted and replaced by tablet bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:06.756942 19190 tablet_service.cc:1514] Processing DeleteTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 with delete_type TABLET_DATA_DELETED (replaced by tablet bb70d5dd96c44b9cac045d923f6d825c) from {username='slave'} at 127.0.0.1:38004
I20260923 17:32:06.757685 19467 tablet_replica.cc:330] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: stopping tablet replica
I20260923 17:32:06.758369 19467 raft_consensus.cc:2229] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:06.758831 19467 raft_consensus.cc:2258] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.760403 19335 tablet_service.cc:1514] Processing DeleteTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 with delete_type TABLET_DATA_DELETED (replaced by tablet bb70d5dd96c44b9cac045d923f6d825c) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.761211 19467 ts_tablet_manager.cc:1905] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 17:32:06.762017 18688 catalog_manager.cc:4821] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): delete failed for tablet b04c5431995d48c7b91f7bfad0ee90a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b04c5431995d48c7b91f7bfad0ee90a6 already in progress: creating tablet
I20260923 17:32:06.751502 19465 consensus_queue.cc:260] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.782991 19460 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:06.783416 19460 ts_tablet_manager.cc:1397] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.040s user 0.013s sys 0.005s
I20260923 17:32:06.783608 19465 ts_tablet_manager.cc:1428] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.177s user 0.039s sys 0.000s
I20260923 17:32:06.785799 19460 raft_consensus.cc:348] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.786477 19460 raft_consensus.cc:374] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.786722 19460 raft_consensus.cc:729] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.787783 18697 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33228:
name: "default.Table_78ce2153804b4feab044ec3a58ea9f38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:06.790297 19467 ts_tablet_manager.cc:1918] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:06.791726 19467 log.cc:1196] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:06.793701 19190 tablet_service.cc:1467] Processing CreateTablet for tablet bb70d5dd96c44b9cac045d923f6d825c (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.794677 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb70d5dd96c44b9cac045d923f6d825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.795061 19335 tablet_service.cc:1467] Processing CreateTablet for tablet bb70d5dd96c44b9cac045d923f6d825c (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.799472 19404 heartbeater.cc:503] Master 127.15.246.190:46265 was elected leader, sending a full tablet report...
I20260923 17:32:06.799739 19045 tablet_service.cc:1467] Processing CreateTablet for tablet bb70d5dd96c44b9cac045d923f6d825c (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:06.800877 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb70d5dd96c44b9cac045d923f6d825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.806649 19335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb70d5dd96c44b9cac045d923f6d825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:06.790874 19460 consensus_queue.cc:260] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.810237 19334 tablet_service.cc:1514] Processing DeleteTablet for tablet b04c5431995d48c7b91f7bfad0ee90a6 with delete_type TABLET_DATA_DELETED (replaced by tablet bb70d5dd96c44b9cac045d923f6d825c) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.811111 19467 ts_tablet_manager.cc:1939] T b04c5431995d48c7b91f7bfad0ee90a6 P 2a823629310d45f880d1c553255a6d1b: Deleting consensus metadata
I20260923 17:32:06.811497 19465 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
W20260923 17:32:06.812798 18688 catalog_manager.cc:4821] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): delete failed for tablet b04c5431995d48c7b91f7bfad0ee90a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b04c5431995d48c7b91f7bfad0ee90a6 already in progress: creating tablet
I20260923 17:32:06.813503 19460 ts_tablet_manager.cc:1428] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.030s user 0.001s sys 0.006s
I20260923 17:32:06.814421 19336 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.816041 19460 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:06.815953 18687 catalog_manager.cc:4841] TS 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): tablet b04c5431995d48c7b91f7bfad0ee90a6 (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]) successfully deleted
W20260923 17:32:06.817848 18688 catalog_manager.cc:4821] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): delete failed for tablet 6440b97f09304b988f67124faacaee58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6440b97f09304b988f67124faacaee58 already in progress: creating tablet
I20260923 17:32:06.823211 19460 tablet_bootstrap.cc:654] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.847894 19465 tablet_bootstrap.cc:654] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
@ 0x7f04face7aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260923 17:32:06.863734 19464 raft_consensus.cc:348] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.871197 19464 raft_consensus.cc:374] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.871533 19464 raft_consensus.cc:729] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.871758 19460 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:06.872160 19460 ts_tablet_manager.cc:1397] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.056s user 0.024s sys 0.024s
I20260923 17:32:06.872320 19465 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:06.872753 19465 ts_tablet_manager.cc:1397] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.004s
I20260923 17:32:06.874500 19460 raft_consensus.cc:348] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.874960 19478 raft_consensus.cc:482] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:06.875227 19460 raft_consensus.cc:374] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.875474 19460 raft_consensus.cc:729] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.875419 19465 raft_consensus.cc:348] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.875468 19478 raft_consensus.cc:504] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.872524 19464 consensus_queue.cc:260] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:06.876304 19465 raft_consensus.cc:374] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.876590 19465 raft_consensus.cc:729] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.876057 19460 consensus_queue.cc:260] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.877285 19465 consensus_queue.cc:260] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
W20260923 17:32:06.884683 19406 tablet.cc:2369] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:32:06.886492 19460 ts_tablet_manager.cc:1428] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.014s user 0.001s sys 0.003s
I20260923 17:32:06.895697 19465 ts_tablet_manager.cc:1428] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260923 17:32:06.897382 19464 ts_tablet_manager.cc:1428] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.171s user 0.043s sys 0.011s
I20260923 17:32:06.901257 19465 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
I20260923 17:32:06.901440 19464 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:06.905318 19335 tablet_service.cc:1514] Processing DeleteTablet for tablet 6440b97f09304b988f67124faacaee58 with delete_type TABLET_DATA_DELETED (replaced by tablet b04c5431995d48c7b91f7bfad0ee90a6) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.906260 19466 tablet_replica.cc:330] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: stopping tablet replica
I20260923 17:32:06.907086 19466 raft_consensus.cc:2229] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:06.907416 19464 tablet_bootstrap.cc:654] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:06.907622 19466 raft_consensus.cc:2258] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.912113 19466 ts_tablet_manager.cc:1905] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:06.914876 19478 leader_election.cc:288] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:06.939251 19465 tablet_bootstrap.cc:654] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
@ 0x7f04face733c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260923 17:32:06.949150 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet e8a06927aa374ac280d04bb5df535d01 from {username='slave'} at 127.0.0.1:33228
@ 0x7f04faceafd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260923 17:32:06.968323 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a06927aa374ac280d04bb5df535d01" candidate_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a823629310d45f880d1c553255a6d1b" is_pre_election: true
I20260923 17:32:06.968598 18696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Sending DeleteTablet for 3 replicas of tablet e8a06927aa374ac280d04bb5df535d01
I20260923 17:32:06.969228 19211 raft_consensus.cc:2454] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f14ff7c89a64a1d9b877b5bfbd443cd in term 0.
I20260923 17:32:06.970052 18696 catalog_manager.cc:6565] ReplaceTablet: tablet e8a06927aa374ac280d04bb5df535d01 deleted and replaced by tablet a5900e799aca43bfa9b0dc1328bb6096
I20260923 17:32:06.972116 19335 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a06927aa374ac280d04bb5df535d01 with delete_type TABLET_DATA_DELETED (replaced by tablet a5900e799aca43bfa9b0dc1328bb6096) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:06.973305 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet bb70d5dd96c44b9cac045d923f6d825c from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:06.973409 19190 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a06927aa374ac280d04bb5df535d01 with delete_type TABLET_DATA_DELETED (replaced by tablet a5900e799aca43bfa9b0dc1328bb6096) from {username='slave'} at 127.0.0.1:38004
I20260923 17:32:06.974148 19467 tablet_replica.cc:330] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: stopping tablet replica
I20260923 17:32:06.974900 19467 raft_consensus.cc:2229] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:06.975332 19467 raft_consensus.cc:2258] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.976608 19289 leader_election.cc:302] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters:
I20260923 17:32:06.977368 19467 ts_tablet_manager.cc:1905] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:06.978487 19465 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:06.978950 19465 ts_tablet_manager.cc:1397] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.078s user 0.006s sys 0.005s
I20260923 17:32:06.979041 19478 raft_consensus.cc:2781] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:06.979387 19478 raft_consensus.cc:482] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:06.979676 19478 raft_consensus.cc:3037] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:06.981633 19465 raft_consensus.cc:348] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.982390 19465 raft_consensus.cc:374] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.982642 19465 raft_consensus.cc:729] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:06.982918 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a06927aa374ac280d04bb5df535d01 with delete_type TABLET_DATA_DELETED (replaced by tablet a5900e799aca43bfa9b0dc1328bb6096) from {username='slave'} at 127.0.0.1:37038
I20260923 17:32:06.983857 19469 tablet_replica.cc:330] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: stopping tablet replica
I20260923 17:32:06.984745 19469 raft_consensus.cc:2229] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:06.985353 19469 raft_consensus.cc:2258] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.988422 19464 tablet_bootstrap.cc:492] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:06.988806 19464 ts_tablet_manager.cc:1397] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.090s user 0.025s sys 0.019s
I20260923 17:32:06.988938 19469 ts_tablet_manager.cc:1905] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:06.991994 19464 raft_consensus.cc:348] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.993436 19464 raft_consensus.cc:374] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:06.992985 19465 consensus_queue.cc:260] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:06.993700 19464 raft_consensus.cc:729] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
@ 0x7f04faceaf8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260923 17:32:06.994858 19466 ts_tablet_manager.cc:1918] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:06.995488 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a06927aa374ac280d04bb5df535d01" candidate_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee" is_pre_election: true
W20260923 17:32:06.997820 19292 leader_election.cc:341] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:32:07.003229 19467 ts_tablet_manager.cc:1918] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:07.003585 19467 log.cc:1196] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/e8a06927aa374ac280d04bb5df535d01
I20260923 17:32:07.004169 19467 ts_tablet_manager.cc:1939] T e8a06927aa374ac280d04bb5df535d01 P 2a823629310d45f880d1c553255a6d1b: Deleting consensus metadata
I20260923 17:32:07.006624 18687 catalog_manager.cc:4841] TS 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): tablet e8a06927aa374ac280d04bb5df535d01 (table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]) successfully deleted
I20260923 17:32:07.006238 19478 raft_consensus.cc:504] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.008769 19478 leader_election.cc:288] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Requested vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:06.995275 19466 log.cc:1196] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/6440b97f09304b988f67124faacaee58
I20260923 17:32:07.009097 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a06927aa374ac280d04bb5df535d01" candidate_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee"
I20260923 17:32:07.009972 19466 ts_tablet_manager.cc:1939] T 6440b97f09304b988f67124faacaee58 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting consensus metadata
W20260923 17:32:07.012326 19289 leader_election.cc:341] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: e8a06927aa374ac280d04bb5df535d01
I20260923 17:32:07.018045 18688 catalog_manager.cc:4841] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): tablet 6440b97f09304b988f67124faacaee58 (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]) successfully deleted
W20260923 17:32:07.018811 19292 leader_election.cc:341] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): Illegal state: Tablet not RUNNING: STOPPED
I20260923 17:32:07.021863 19292 leader_election.cc:302] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters: 2a823629310d45f880d1c553255a6d1b, dedc4fa600d8426c9074e3c8916aeaee
I20260923 17:32:07.022755 19478 raft_consensus.cc:2726] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 17:32:06.997524 19465 ts_tablet_manager.cc:1428] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.018s user 0.003s sys 0.005s
I20260923 17:32:07.018307 19466 tablet_replica.cc:330] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: stopping tablet replica
I20260923 17:32:07.023854 19466 raft_consensus.cc:2229] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:07.024379 19466 raft_consensus.cc:2258] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:07.024634 18696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca93f1f37ccd46c08821589340bf34e1: Sending DeleteTablet for 3 replicas of tablet bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.025704 19465 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
I20260923 17:32:07.026554 19466 ts_tablet_manager.cc:1905] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:07.026648 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet bb70d5dd96c44b9cac045d923f6d825c with delete_type TABLET_DATA_DELETED (replaced by tablet beb32e39cda941f1ae80851799093c49) from {username='slave'} at 127.0.0.1:37038
I20260923 17:32:07.033444 19465 tablet_bootstrap.cc:654] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.039726 19335 tablet_service.cc:1514] Processing DeleteTablet for tablet bb70d5dd96c44b9cac045d923f6d825c with delete_type TABLET_DATA_DELETED (replaced by tablet beb32e39cda941f1ae80851799093c49) from {username='slave'} at 127.0.0.1:49784
I20260923 17:32:07.040920 19469 ts_tablet_manager.cc:1918] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:07.041332 19469 log.cc:1196] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-0/wal/wals/e8a06927aa374ac280d04bb5df535d01
I20260923 17:32:07.042141 19469 ts_tablet_manager.cc:1939] T e8a06927aa374ac280d04bb5df535d01 P dedc4fa600d8426c9074e3c8916aeaee: Deleting consensus metadata
I20260923 17:32:07.046720 18690 catalog_manager.cc:4841] TS dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): tablet e8a06927aa374ac280d04bb5df535d01 (table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]) successfully deleted
W20260923 17:32:07.048933 18690 catalog_manager.cc:4821] TS dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): delete failed for tablet bb70d5dd96c44b9cac045d923f6d825c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb70d5dd96c44b9cac045d923f6d825c already in progress: creating tablet
I20260923 17:32:07.053421 19045 tablet_service.cc:1467] Processing CreateTablet for tablet a5900e799aca43bfa9b0dc1328bb6096 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.054267 19190 tablet_service.cc:1467] Processing CreateTablet for tablet a5900e799aca43bfa9b0dc1328bb6096 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.054540 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5900e799aca43bfa9b0dc1328bb6096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:07.055027 18696 catalog_manager.cc:6565] ReplaceTablet: tablet bb70d5dd96c44b9cac045d923f6d825c deleted and replaced by tablet beb32e39cda941f1ae80851799093c49
I20260923 17:32:07.055233 19190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5900e799aca43bfa9b0dc1328bb6096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:07.055799 19335 tablet_service.cc:1467] Processing CreateTablet for tablet a5900e799aca43bfa9b0dc1328bb6096 (DEFAULT_TABLE table=default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.056916 19335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5900e799aca43bfa9b0dc1328bb6096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:07.056927 19191 tablet_service.cc:1514] Processing DeleteTablet for tablet bb70d5dd96c44b9cac045d923f6d825c with delete_type TABLET_DATA_DELETED (replaced by tablet beb32e39cda941f1ae80851799093c49) from {username='slave'} at 127.0.0.1:38004
I20260923 17:32:07.057601 19467 tablet_replica.cc:330] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: stopping tablet replica
I20260923 17:32:07.058286 19467 raft_consensus.cc:2229] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260923 17:32:07.058712 19467 raft_consensus.cc:2258] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260923 17:32:06.994437 19464 consensus_queue.cc:260] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.060704 19467 ts_tablet_manager.cc:1905] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 17:32:07.063550 19482 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.064064 19482 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.092458 19482 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
@ 0x7f04faceaf1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260923 17:32:07.107609 19045 tablet_service.cc:1467] Processing CreateTablet for tablet beb32e39cda941f1ae80851799093c49 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.108721 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb32e39cda941f1ae80851799093c49. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:32:07.110689 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet 6440b97f09304b988f67124faacaee58
I20260923 17:32:07.114818 19191 tablet_service.cc:1467] Processing CreateTablet for tablet beb32e39cda941f1ae80851799093c49 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.116034 19191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb32e39cda941f1ae80851799093c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 17:32:07.130648 19465 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:07.131099 19465 ts_tablet_manager.cc:1397] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.106s user 0.017s sys 0.030s
I20260923 17:32:07.135190 19465 raft_consensus.cc:348] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.135911 19465 raft_consensus.cc:374] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.136170 19465 raft_consensus.cc:729] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.141407 19466 ts_tablet_manager.cc:1918] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:07.141832 19466 log.cc:1196] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-2/wal/wals/e8a06927aa374ac280d04bb5df535d01
I20260923 17:32:07.142529 19466 ts_tablet_manager.cc:1939] T e8a06927aa374ac280d04bb5df535d01 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Deleting consensus metadata
I20260923 17:32:07.144565 19465 consensus_queue.cc:260] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.147039 19335 tablet_service.cc:1467] Processing CreateTablet for tablet beb32e39cda941f1ae80851799093c49 (DEFAULT_TABLE table=default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 17:32:07.148564 19335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb32e39cda941f1ae80851799093c49. 1 dirs total, 0 dirs full, 0 dirs failed
W20260923 17:32:07.151878 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:07.152838 19463 raft_consensus.cc:482] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.153381 19463 raft_consensus.cc:504] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.154665 19464 ts_tablet_manager.cc:1428] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.166s user 0.008s sys 0.006s
I20260923 17:32:07.155766 19464 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
W20260923 17:32:07.160511 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet 6440b97f09304b988f67124faacaee58
I20260923 17:32:07.161638 19463 leader_election.cc:288] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.161373 19465 ts_tablet_manager.cc:1428] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.030s user 0.006s sys 0.001s
W20260923 17:32:07.176961 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:07.179157 19465 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
I20260923 17:32:07.182858 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet b04c5431995d48c7b91f7bfad0ee90a6 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:07.183411 19464 tablet_bootstrap.cc:654] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.186515 19467 ts_tablet_manager.cc:1918] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260923 17:32:07.186896 19467 log.cc:1196] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Deleting WAL directory at /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/ts-1/wal/wals/bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.187532 19467 ts_tablet_manager.cc:1939] T bb70d5dd96c44b9cac045d923f6d825c P 2a823629310d45f880d1c553255a6d1b: Deleting consensus metadata
I20260923 17:32:07.196151 19460 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:07.198522 18688 catalog_manager.cc:4841] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): tablet e8a06927aa374ac280d04bb5df535d01 (table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]) successfully deleted
I20260923 17:32:07.202088 19465 tablet_bootstrap.cc:654] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.202497 19460 tablet_bootstrap.cc:654] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
W20260923 17:32:07.208755 18688 catalog_manager.cc:4821] TS 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): delete failed for tablet bb70d5dd96c44b9cac045d923f6d825c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb70d5dd96c44b9cac045d923f6d825c already in progress: creating tablet
@ 0x7f04faceaee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260923 17:32:07.219693 19465 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:07.220726 19465 ts_tablet_manager.cc:1397] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.010s
I20260923 17:32:07.222019 19460 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:07.222481 19460 ts_tablet_manager.cc:1397] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.005s
I20260923 17:32:07.223625 19465 raft_consensus.cc:348] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.224383 19465 raft_consensus.cc:374] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.220263 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6753a0b86b7646339cec60f5c47811f1" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee" is_pre_election: true
I20260923 17:32:07.224627 19465 raft_consensus.cc:729] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
W20260923 17:32:07.224901 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:07.225513 19465 consensus_queue.cc:260] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
W20260923 17:32:07.226241 19146 leader_election.cc:341] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787): Illegal state: must be running to vote when last-logged opid is not known
I20260923 17:32:07.233573 19465 ts_tablet_manager.cc:1428] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260923 17:32:07.234562 19465 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
I20260923 17:32:07.234213 19460 raft_consensus.cc:348] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
W20260923 17:32:07.234894 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.235015 19460 raft_consensus.cc:374] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.235270 19460 raft_consensus.cc:729] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.237339 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6753a0b86b7646339cec60f5c47811f1" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" is_pre_election: true
I20260923 17:32:07.238145 19356 raft_consensus.cc:2454] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 0.
I20260923 17:32:07.240118 19144 leader_election.cc:302] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters: dedc4fa600d8426c9074e3c8916aeaee
I20260923 17:32:07.241371 19463 raft_consensus.cc:2781] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:07.241720 19463 raft_consensus.cc:482] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:07.242331 19463 raft_consensus.cc:3037] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.244939 19460 consensus_queue.cc:260] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.250250 19465 tablet_bootstrap.cc:654] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
W20260923 17:32:07.253890 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:07.254364 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
W20260923 17:32:07.255188 18696 catalog_manager.cc:5328] Ignoring report from unknown tablet bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.256630 19494 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:07.259003 19464 tablet_bootstrap.cc:492] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:07.258857 19463 raft_consensus.cc:504] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.259558 19464 ts_tablet_manager.cc:1397] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.104s user 0.005s sys 0.011s
I20260923 17:32:07.262550 19464 raft_consensus.cc:348] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.263360 19464 raft_consensus.cc:374] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.263645 19464 raft_consensus.cc:729] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.266616 19463 leader_election.cc:288] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Requested vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.266633 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6753a0b86b7646339cec60f5c47811f1" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee"
I20260923 17:32:07.268174 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6753a0b86b7646339cec60f5c47811f1" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
I20260923 17:32:07.268761 19356 raft_consensus.cc:3037] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.270238 19460 ts_tablet_manager.cc:1428] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.047s user 0.007s sys 0.002s
I20260923 17:32:07.274355 19460 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b: Bootstrap starting.
I20260923 17:32:07.277446 18687 catalog_manager.cc:4841] TS 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): tablet bb70d5dd96c44b9cac045d923f6d825c (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]) successfully deleted
I20260923 17:32:07.279961 18696 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:33228
I20260923 17:32:07.278533 19464 consensus_queue.cc:260] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.281843 19065 raft_consensus.cc:3037] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.285444 19356 raft_consensus.cc:2454] T 6753a0b86b7646339cec60f5c47811f1 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.286495 19144 leader_election.cc:302] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters:
I20260923 17:32:07.287281 19463 raft_consensus.cc:2781] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:32:07.287829 19463 raft_consensus.cc:686] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [term 1 LEADER]: Becoming Leader. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Running, Role: LEADER
@ 0x7f04facea1de std::__1::__function::__func<>::operator()() at ??:0
I20260923 17:32:07.289726 19065 raft_consensus.cc:2454] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.292102 19463 consensus_queue.cc:237] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
I20260923 17:32:07.295782 19464 ts_tablet_manager.cc:1428] T 6753a0b86b7646339cec60f5c47811f1 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.036s user 0.004s sys 0.003s
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260923 17:32:07.310596 19465 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:07.311077 19465 ts_tablet_manager.cc:1397] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.077s user 0.013s sys 0.018s
I20260923 17:32:07.312265 19460 tablet_bootstrap.cc:654] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b: Neither blocks nor log segments found. Creating new log.
W20260923 17:32:07.313761 19118 tablet.cc:2369] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 17:32:07.315173 19465 raft_consensus.cc:348] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.316067 19465 raft_consensus.cc:374] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.316287 19464 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:07.316457 19465 raft_consensus.cc:729] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.320523 19465 consensus_queue.cc:260] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
@ 0x7f04f1dfddc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260923 17:32:07.324535 19464 tablet_bootstrap.cc:654] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.328505 18696 catalog_manager.cc:5495] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a823629310d45f880d1c553255a6d1b (127.15.246.130). New cstate: current_term: 1 leader_uuid: "2a823629310d45f880d1c553255a6d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } health_report { overall_health: HEALTHY } } }
I20260923 17:32:07.339509 19465 ts_tablet_manager.cc:1428] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.028s user 0.006s sys 0.003s
@ 0x7f04f346b6db start_thread at ??:0
I20260923 17:32:07.363514 19465 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Bootstrap starting.
@ 0x7f04ee8f161f clone at ??:0
I20260923 17:32:07.371206 19465 tablet_bootstrap.cc:654] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.374404 19460 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b: No bootstrap required, opened a new log
I20260923 17:32:07.374918 19460 ts_tablet_manager.cc:1397] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.012s
I20260923 17:32:07.377655 19460 raft_consensus.cc:348] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.378319 19460 raft_consensus.cc:374] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.378537 19460 raft_consensus.cc:729] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.379508 19460 consensus_queue.cc:260] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.381470 19460 ts_tablet_manager.cc:1428] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260923 17:32:07.389164 19464 tablet_bootstrap.cc:492] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:07.389623 19464 ts_tablet_manager.cc:1397] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.074s user 0.013s sys 0.003s
I20260923 17:32:07.396955 19464 raft_consensus.cc:348] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.397796 19464 raft_consensus.cc:374] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.398053 19464 raft_consensus.cc:729] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.404796 19465 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd: No bootstrap required, opened a new log
I20260923 17:32:07.405232 19465 ts_tablet_manager.cc:1397] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.005s
I20260923 17:32:07.407429 19465 raft_consensus.cc:348] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.408035 19465 raft_consensus.cc:374] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.408316 19465 raft_consensus.cc:729] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.408951 19465 consensus_queue.cc:260] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.415715 19464 consensus_queue.cc:260] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.418488 19464 ts_tablet_manager.cc:1428] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260923 17:32:07.422731 19464 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:07.429085 19464 tablet_bootstrap.cc:654] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.436704 19465 ts_tablet_manager.cc:1428] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260923 17:32:07.461706 19464 tablet_bootstrap.cc:492] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:07.462185 19464 ts_tablet_manager.cc:1397] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.040s user 0.016s sys 0.000s
I20260923 17:32:07.464833 19464 raft_consensus.cc:348] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.465554 19464 raft_consensus.cc:374] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.465827 19464 raft_consensus.cc:729] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.466498 19464 consensus_queue.cc:260] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.475754 19464 ts_tablet_manager.cc:1428] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.013s user 0.005s sys 0.002s
I20260923 17:32:07.476845 19464 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:07.483044 19464 tablet_bootstrap.cc:654] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.511904 19464 tablet_bootstrap.cc:492] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:07.512429 19464 ts_tablet_manager.cc:1397] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.007s
I20260923 17:32:07.514992 19464 raft_consensus.cc:348] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.515714 19464 raft_consensus.cc:374] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.515967 19464 raft_consensus.cc:729] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.516748 19464 consensus_queue.cc:260] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.519455 19464 ts_tablet_manager.cc:1428] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260923 17:32:07.520557 19464 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee: Bootstrap starting.
I20260923 17:32:07.530089 19464 tablet_bootstrap.cc:654] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee: Neither blocks nor log segments found. Creating new log.
I20260923 17:32:07.537137 19464 tablet_bootstrap.cc:492] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee: No bootstrap required, opened a new log
I20260923 17:32:07.537686 19464 ts_tablet_manager.cc:1397] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s
I20260923 17:32:07.540630 19464 raft_consensus.cc:348] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.541323 19464 raft_consensus.cc:374] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 17:32:07.541563 19464 raft_consensus.cc:729] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Initialized, Role: FOLLOWER
I20260923 17:32:07.542244 19464 consensus_queue.cc:260] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.546563 19464 ts_tablet_manager.cc:1428] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260923 17:32:07.582113 19463 raft_consensus.cc:482] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.582576 19463 raft_consensus.cc:504] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.584540 19463 leader_election.cc:288] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.587000 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb32e39cda941f1ae80851799093c49" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee" is_pre_election: true
I20260923 17:32:07.587677 19065 raft_consensus.cc:2454] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 0.
I20260923 17:32:07.588558 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb32e39cda941f1ae80851799093c49" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" is_pre_election: true
I20260923 17:32:07.589138 19356 raft_consensus.cc:2454] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 0.
I20260923 17:32:07.590616 19144 leader_election.cc:302] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters:
I20260923 17:32:07.592265 19463 raft_consensus.cc:2781] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:07.592624 19463 raft_consensus.cc:482] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:07.592990 19463 raft_consensus.cc:3037] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.599038 19463 raft_consensus.cc:504] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.607559 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb32e39cda941f1ae80851799093c49" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee"
I20260923 17:32:07.608131 19065 raft_consensus.cc:3037] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.612762 19505 raft_consensus.cc:482] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.613207 19505 raft_consensus.cc:504] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.615370 19505 leader_election.cc:288] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.617069 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5900e799aca43bfa9b0dc1328bb6096" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" is_pre_election: true
I20260923 17:32:07.617651 19356 raft_consensus.cc:2454] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 0.
I20260923 17:32:07.619421 19064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5900e799aca43bfa9b0dc1328bb6096" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee" is_pre_election: true
I20260923 17:32:07.619537 19144 leader_election.cc:302] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters:
I20260923 17:32:07.620002 19064 raft_consensus.cc:2454] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 0.
I20260923 17:32:07.620500 19505 raft_consensus.cc:2781] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:07.620844 19505 raft_consensus.cc:482] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:07.621199 19505 raft_consensus.cc:3037] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.622605 19065 raft_consensus.cc:2454] T beb32e39cda941f1ae80851799093c49 P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.623592 19146 leader_election.cc:302] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, dedc4fa600d8426c9074e3c8916aeaee; no voters:
I20260923 17:32:07.627635 19505 raft_consensus.cc:504] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.631047 19463 leader_election.cc:288] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Requested vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.631076 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5900e799aca43bfa9b0dc1328bb6096" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
I20260923 17:32:07.631522 19463 raft_consensus.cc:2781] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:32:07.631564 19356 raft_consensus.cc:3037] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.631897 19463 raft_consensus.cc:686] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [term 1 LEADER]: Becoming Leader. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Running, Role: LEADER
I20260923 17:32:07.632635 19463 consensus_queue.cc:237] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.633205 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb32e39cda941f1ae80851799093c49" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
I20260923 17:32:07.633711 19355 raft_consensus.cc:3037] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.640154 19494 raft_consensus.cc:482] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.640718 19494 raft_consensus.cc:504] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.642742 19494 leader_election.cc:288] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:07.644048 19354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04c5431995d48c7b91f7bfad0ee90a6" candidate_uuid: "dedc4fa600d8426c9074e3c8916aeaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" is_pre_election: true
I20260923 17:32:07.644806 19354 raft_consensus.cc:2454] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dedc4fa600d8426c9074e3c8916aeaee in term 0.
I20260923 17:32:07.645533 19356 raft_consensus.cc:2454] T a5900e799aca43bfa9b0dc1328bb6096 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.646292 19355 raft_consensus.cc:2454] T beb32e39cda941f1ae80851799093c49 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.646996 18997 leader_election.cc:302] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f14ff7c89a64a1d9b877b5bfbd443cd, dedc4fa600d8426c9074e3c8916aeaee; no voters:
I20260923 17:32:07.647714 19505 leader_election.cc:288] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Requested vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:07.648108 19144 leader_election.cc:302] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd; no voters:
W20260923 17:32:07.650035 18996 leader_election.cc:341] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:07.651530 19494 raft_consensus.cc:2781] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:07.651866 19494 raft_consensus.cc:482] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:07.651530 19505 raft_consensus.cc:2781] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:32:07.652172 19494 raft_consensus.cc:3037] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.652809 19505 raft_consensus.cc:686] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [term 1 LEADER]: Becoming Leader. State: Replica: 2a823629310d45f880d1c553255a6d1b, State: Running, Role: LEADER
I20260923 17:32:07.656126 19505 consensus_queue.cc:237] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } }
I20260923 17:32:07.658634 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5900e799aca43bfa9b0dc1328bb6096" candidate_uuid: "2a823629310d45f880d1c553255a6d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee"
I20260923 17:32:07.659163 19065 raft_consensus.cc:3037] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:32:07.659334 19471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:32:07.660343 19471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:32:07.661056 19471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 17:32:07.662503 19494 raft_consensus.cc:504] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.664456 19494 leader_election.cc:288] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 election: Requested vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:07.666353 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04c5431995d48c7b91f7bfad0ee90a6" candidate_uuid: "dedc4fa600d8426c9074e3c8916aeaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd"
I20260923 17:32:07.666918 19355 raft_consensus.cc:3037] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:32:07.667856 18996 leader_election.cc:341] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:07.671469 19065 raft_consensus.cc:2454] T a5900e799aca43bfa9b0dc1328bb6096 P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a823629310d45f880d1c553255a6d1b in term 1.
I20260923 17:32:07.672905 19355 raft_consensus.cc:2454] T b04c5431995d48c7b91f7bfad0ee90a6 P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dedc4fa600d8426c9074e3c8916aeaee in term 1.
I20260923 17:32:07.673961 18997 leader_election.cc:302] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f14ff7c89a64a1d9b877b5bfbd443cd, dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b
I20260923 17:32:07.674803 19494 raft_consensus.cc:2781] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:32:07.677917 19494 raft_consensus.cc:686] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [term 1 LEADER]: Becoming Leader. State: Replica: dedc4fa600d8426c9074e3c8916aeaee, State: Running, Role: LEADER
I20260923 17:32:07.678946 19494 consensus_queue.cc:237] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
W20260923 17:32:07.719864 19471 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260923 17:32:07.720432 19471 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260923 17:32:07.720794 19471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:32:07.721138 19471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:32:07.721477 19471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:32:07.721850 19471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:32:07.770354 19471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38515
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.188:41441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.246.188
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
W20260923 17:32:07.771705 19290 connection.cc:535] client connection to 127.15.246.190:46265 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:07.771850 19144 connection.cc:535] client connection to 127.15.246.190:46265 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:07.771778 19413 connection.cc:535] client connection to 127.15.246.190:46265 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:32:07.773476 19471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260923 17:32:07.774250 19413 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.15.246.190:46265)
W20260923 17:32:07.775354 18997 connection.cc:535] client connection to 127.15.246.190:46265 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:07.776724 19487 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 17:32:07.773488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.246.190:46265 (local address 127.15.246.131:59703)
0923 17:32:07.774285 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0923 17:32:07.774328 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 17:32:07.774569 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 17:32:07.775591 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":374}
I20260923 17:32:07.776894 19471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:32:07.779373 19413 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.246.190:46265)
W20260923 17:32:07.781466 19413 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.15.246.190:46265)
W20260923 17:32:07.783972 19413 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.15.246.190:46265)
W20260923 17:32:07.805706 19533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:07.853747 19539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca93f1f37ccd46c08821589340bf34e1)
I20260923 17:32:07.854300 19539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:07.855888 19539 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:07.861719 18763 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:07.864676 18762 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:07.864995 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:07.865523 19539 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260923 17:32:07.915071 19478 raft_consensus.cc:482] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:07.915792 19478 raft_consensus.cc:504] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.918154 19478 leader_election.cc:288] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:07.919288 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb70d5dd96c44b9cac045d923f6d825c" candidate_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee" is_pre_election: true
W20260923 17:32:07.919978 19289 leader_election.cc:341] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.920101 19065 raft_consensus.cc:2454] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f14ff7c89a64a1d9b877b5bfbd443cd in term 0.
I20260923 17:32:07.921808 19292 leader_election.cc:302] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f14ff7c89a64a1d9b877b5bfbd443cd, dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b
I20260923 17:32:07.923281 19478 raft_consensus.cc:2781] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 17:32:07.923710 19478 raft_consensus.cc:482] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 17:32:07.924041 19478 raft_consensus.cc:3037] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 17:32:07.932367 19478 raft_consensus.cc:504] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:07.935318 19478 leader_election.cc:288] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Requested vote from peers dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:07.935521 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb70d5dd96c44b9cac045d923f6d825c" candidate_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc4fa600d8426c9074e3c8916aeaee"
I20260923 17:32:07.936121 19065 raft_consensus.cc:3037] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Advancing to term 1
W20260923 17:32:07.936656 19289 leader_election.cc:341] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c
I20260923 17:32:07.943266 19065 raft_consensus.cc:2454] T bb70d5dd96c44b9cac045d923f6d825c P dedc4fa600d8426c9074e3c8916aeaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f14ff7c89a64a1d9b877b5bfbd443cd in term 1.
I20260923 17:32:07.944350 19292 leader_election.cc:302] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f14ff7c89a64a1d9b877b5bfbd443cd, dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b
I20260923 17:32:07.945363 19478 raft_consensus.cc:2781] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 17:32:07.946800 19478 raft_consensus.cc:686] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [term 1 LEADER]: Becoming Leader. State: Replica: 7f14ff7c89a64a1d9b877b5bfbd443cd, State: Running, Role: LEADER
I20260923 17:32:07.950626 19478 consensus_queue.cc:237] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:08.342216 19546 consensus_queue.cc:1035] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:32:08.377578 19546 consensus_queue.cc:1035] T 6753a0b86b7646339cec60f5c47811f1 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:32:08.438369 19553 consensus_queue.cc:1035] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260923 17:32:08.454736 19549 consensus_queue.cc:1035] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260923 17:32:08.599462 19559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca93f1f37ccd46c08821589340bf34e1)
I20260923 17:32:08.600281 19559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:08.602934 19559 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:08.604708 19547 consensus_queue.cc:1035] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:32:08.613618 19549 consensus_queue.cc:1035] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 17:32:08.656840 19546 consensus_queue.cc:1035] T a5900e799aca43bfa9b0dc1328bb6096 P 2a823629310d45f880d1c553255a6d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:32:08.667670 18762 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:32:08.685426 18763 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:08.686033 18763 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:08.688918 19559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260923 17:32:08.704301 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 1: this message will repeat every 5th retry.
W20260923 17:32:08.910033 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 1: this message will repeat every 5th retry.
I20260923 17:32:08.998178 19478 consensus_queue.cc:1035] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 17:32:07.806587 19536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 17:32:07.806854 19532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:07.807739 19471 server_base.cc:1034] running on GCE node
I20260923 17:32:09.071554 19471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:32:09.074654 19471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:32:09.076081 19471 hybrid_clock.cc:648] HybridClock initialized: now 1790184729076022 us; error 76 us; skew 500 ppm
I20260923 17:32:09.077155 19471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:32:09.084493 19471 webserver.cc:460] Webserver started at http://127.15.246.188:41541/ using document root <none> and password file <none>
I20260923 17:32:09.085575 19471 fs_manager.cc:359] Metadata directory not provided
I20260923 17:32:09.085848 19471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:32:09.095402 19471 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 17:32:09.101610 19576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:09.103041 19471 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260923 17:32:09.103492 19471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
uuid: "469a88f1fddf4ecba558f342a2fe37fd"
format_stamp: "Formatted at 2026-09-23 17:31:57 on dist-test-slave-mk2q"
I20260923 17:32:09.106349 19471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:32:09.155118 19471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:32:09.156896 19471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:32:09.157382 19471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:32:09.247872 19471 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.188:41441
I20260923 17:32:09.247960 19630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.188:41441 every 8 connection(s)
I20260923 17:32:09.252367 19471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-2/data/info.pb
I20260923 17:32:09.253438 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 19471
I20260923 17:32:09.265848 19632 sys_catalog.cc:263] Verifying existing consensus state
I20260923 17:32:09.273278 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Bootstrap starting.
I20260923 17:32:09.289384 19632 log.cc:824] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Log is configured to *not* fsync() on all Append() calls
I20260923 17:32:09.305255 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 17:32:09.306236 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Bootstrap complete.
I20260923 17:32:09.327132 19632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:09.330572 19632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469a88f1fddf4ecba558f342a2fe37fd, State: Initialized, Role: FOLLOWER
I20260923 17:32:09.331336 19632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:09.334244 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.335166 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:09.335745 19632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:32:09.373183 19652 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:32:09.373492 19652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: loading cluster ID for follower catalog manager: success
I20260923 17:32:09.382323 19652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: acquiring CA information for follower catalog manager: success
I20260923 17:32:09.398432 19652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:32:09.419802 19657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ca93f1f37ccd46c08821589340bf34e1)
I20260923 17:32:09.420305 19657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:09.370287 19471 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:09.457737 19471 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:09.464931 19256 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:09.483259 19657 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:09.488160 19594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "2a823629310d45f880d1c553255a6d1b" instance_seqno: 1790184723449243) as {username='slave'} at 127.15.246.130:54431; Asking this server to re-register.
I20260923 17:32:09.496770 19256 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:09.501979 19595 ts_manager.cc:194] Registered new tserver with Master: 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:09.510535 18763 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:09.510408 19605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 3 candidate_status { last_received { term: 2 index: 21 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd" is_pre_election: true
I20260923 17:32:09.511379 19605 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a13f8d28964563b4072f7542fd178b in term 1.
I20260923 17:32:09.518990 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd; no voters: ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:09.519945 19657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 17:32:09.520308 19657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ca93f1f37ccd46c08821589340bf34e1)
I20260923 17:32:09.520627 19657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 2 FOLLOWER]: Advancing to term 3
I20260923 17:32:09.539554 19402 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:09.540181 19402 heartbeater.cc:503] Master 127.15.246.188:41441 was elected leader, sending a full tablet report...
I20260923 17:32:09.540110 19657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:09.546252 19594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" instance_seqno: 1790184725330536) as {username='slave'} at 127.15.246.131:34237; Asking this server to re-register.
I20260923 17:32:09.550805 19402 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:09.553582 19110 heartbeater.cc:347] Connected to a master server at 127.15.246.188:41441
I20260923 17:32:09.555437 19657 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 election: Requested vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
I20260923 17:32:09.556998 19605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a13f8d28964563b4072f7542fd178b" candidate_term: 3 candidate_status { last_received { term: 2 index: 21 } } ignore_live_leader: false dest_uuid: "469a88f1fddf4ecba558f342a2fe37fd"
I20260923 17:32:09.557579 19605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 1 FOLLOWER]: Advancing to term 3
I20260923 17:32:09.557679 19594 ts_manager.cc:194] Registered new tserver with Master: 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
W20260923 17:32:09.560813 18763 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:09.568130 19605 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a13f8d28964563b4072f7542fd178b in term 3.
I20260923 17:32:09.572499 19595 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" instance_seqno: 1790184721395473) as {username='slave'} at 127.15.246.129:55577; Asking this server to re-register.
I20260923 17:32:09.574182 18762 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd; no voters: ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:09.576273 19657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Leader election won for term 3
I20260923 17:32:09.577313 19110 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:09.582209 19657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 LEADER]: Becoming Leader. State: Replica: 05a13f8d28964563b4072f7542fd178b, State: Running, Role: LEADER
I20260923 17:32:09.583392 19657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 2.21, Last appended by leader: 21, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:09.584661 19594 ts_manager.cc:194] Registered new tserver with Master: dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787)
I20260923 17:32:09.603396 19667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05a13f8d28964563b4072f7542fd178b. Latest consensus state: current_term: 3 leader_uuid: "05a13f8d28964563b4072f7542fd178b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.604775 19667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: LEADER
I20260923 17:32:09.609176 19669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:32:09.618253 19669 catalog_manager.cc:669] Loaded metadata for table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]
I20260923 17:32:09.620551 19669 catalog_manager.cc:669] Loaded metadata for table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]
I20260923 17:32:09.622306 19669 catalog_manager.cc:669] Loaded metadata for table default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153]
I20260923 17:32:09.629961 19669 tablet_loader.cc:96] loaded metadata for tablet 6753a0b86b7646339cec60f5c47811f1 (table default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153])
I20260923 17:32:09.631225 19669 tablet_loader.cc:96] loaded metadata for tablet a5900e799aca43bfa9b0dc1328bb6096 (table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a])
I20260923 17:32:09.632284 19669 tablet_loader.cc:96] loaded metadata for tablet beb32e39cda941f1ae80851799093c49 (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db])
I20260923 17:32:09.633160 19669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 17:32:09.636931 19669 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:32:09.637108 19669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:32:09.641075 19669 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:32:09.644200 19669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Loaded TSK: 0
I20260923 17:32:09.645797 19669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 17:32:09.650259 19669 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260923 17:32:09.655069 19669 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260923 17:32:09.660118 18777 catalog_manager.cc:5495] T beb32e39cda941f1ae80851799093c49 P 2a823629310d45f880d1c553255a6d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a823629310d45f880d1c553255a6d1b (127.15.246.130). New cstate: current_term: 1 leader_uuid: "2a823629310d45f880d1c553255a6d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } health_report { overall_health: HEALTHY } } }
I20260923 17:32:09.667002 18776 master_service.cc:946] ReplaceTablet: received request to replace tablet bb70d5dd96c44b9cac045d923f6d825c from {username='slave'} at 127.0.0.1:42448
I20260923 17:32:09.671294 18776 master_service.cc:946] ReplaceTablet: received request to replace tablet bb70d5dd96c44b9cac045d923f6d825c from {username='slave'} at 127.0.0.1:42448
I20260923 17:32:09.673537 18772 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42448:
table { table_name: "default.Table_78b2f53459124b318a200b9c57216ec9" } new_table_name: "default.Table_029ff3fda0d747e68a515ed3dd98d6e4" modify_external_catalogs: true
I20260923 17:32:09.673122 18776 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42448:
table { table_name: "default.Table_e6ca980ce29c4245bb88eaf5eb38907b" } new_table_name: "default.Table_fd2cd848c438499980de5e4bae0b10b6" modify_external_catalogs: true
I20260923 17:32:09.668614 18775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42448:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
I20260923 17:32:09.672973 18773 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42448:
name: "default.Table_0ba166e019864d25946c15765c3f0b71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:09.671235 18774 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42448:
name: "default.Table_af14483dad1b4c50a7033d85e9cbc651"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:09.675051 18771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42448:
name: "default.Table_6a36064c5aba4b21bd8b5da2ccb2212e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:09.668838 18778 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42448:
name: "default.Table_78ce2153804b4feab044ec3a58ea9f38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:09.695621 18775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42448:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
I20260923 17:32:09.706029 19605 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Refusing update from remote peer 05a13f8d28964563b4072f7542fd178b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 23. (index mismatch)
I20260923 17:32:09.707844 19657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [LEADER]: Connected to new peer: Peer: permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 4, Time since last communication: 0.001s
W20260923 17:32:09.710911 18763 consensus_peers.cc:487] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b -> Peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Couldn't send request to peer ca93f1f37ccd46c08821589340bf34e1. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260923 17:32:09.714177 18965 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_319978fc115b4d10b8f8f3e8fac76165
*** Check failure stack trace: ***
@ 0x7f5e6f2d19a0 google::LogMessage::Flush() at ??:0
@ 0x7f5e6f2d170f google::LogMessage::~LogMessage() at ??:0
I20260923 17:32:09.765283 19673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "05a13f8d28964563b4072f7542fd178b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.766273 19673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:09.771807 19670 mvcc.cc:205] Tried to move back new op lower bound from 7332596652854112256 to 7332596652384624640. Current Snapshot: MvccSnapshot[applied={T|T < 7332596652854112256}]
I20260923 17:32:09.772969 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05a13f8d28964563b4072f7542fd178b. Latest consensus state: current_term: 3 leader_uuid: "05a13f8d28964563b4072f7542fd178b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.773802 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5e7a36caea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260923 17:32:09.778688 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "05a13f8d28964563b4072f7542fd178b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.779402 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:09.780788 19657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "05a13f8d28964563b4072f7542fd178b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:09.781529 19657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: LEADER
I20260923 17:32:09.799710 19671 mvcc.cc:205] Tried to move back new op lower bound from 7332596652854112256 to 7332596637197012992. Current Snapshot: MvccSnapshot[applied={T|T < 7332596643173232640}]
@ 0x7f5e7a36c33c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f5e7a36ffd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f5e7a36ff8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
@ 0x7f5e7a36ff1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260923 17:32:09.879992 18777 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:42448
I20260923 17:32:09.883008 18777 master_service.cc:946] ReplaceTablet: received request to replace tablet bb70d5dd96c44b9cac045d923f6d825c from {username='slave'} at 127.0.0.1:42448
@ 0x7f5e7a36fee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
W20260923 17:32:09.891657 18810 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6ae0 after lost signal to thread 18744
W20260923 17:32:09.892367 18810 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5f00 after lost signal to thread 18813
@ 0x7f5e7a36f1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
@ 0x7f5e71482dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
@ 0x7f5e72af06db start_thread at ??:0
@ 0x7f5e6df7661f clone at ??:0
W20260923 17:32:10.153434 19292 connection.cc:535] client connection to 127.15.246.189:35267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:10.153554 19415 connection.cc:535] client connection to 127.15.246.189:35267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:10.155468 19415 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.15.246.189:35267)
W20260923 17:32:10.159065 19415 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.246.189:35267)
W20260923 17:32:10.159778 19415 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.15.246.189:35267)
W20260923 17:32:10.161728 19415 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.15.246.189:35267)
W20260923 17:32:10.163295 18999 connection.cc:535] client connection to 127.15.246.189:35267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 17:32:10.277190 19637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:10.277897 19637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:10.281668 19637 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
W20260923 17:32:10.284806 19578 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:10.288897 19258 heartbeater.cc:503] Master 127.15.246.189:35267 was elected leader, sending a full tablet report...
W20260923 17:32:10.302448 19580 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267): Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:32:10.303262 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:10.305395 19578 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd; no voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:10.306382 19637 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260923 17:32:10.473950 19699 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:10.474493 19699 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:10.476336 19699 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:10.478130 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:32:10.478860 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:10.479264 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:10.480108 19699 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:10.918735 19709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:10.919211 19709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:10.921010 19709 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
W20260923 17:32:10.926443 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:10.927428 19580 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267): Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
I20260923 17:32:10.927899 19580 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd; no voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:10.928689 19709 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260923 17:32:11.367447 19626 debug-util.cc:398] Leaking SignalData structure 0x7b08000720a0 after lost signal to thread 19521
W20260923 17:32:11.368316 19626 debug-util.cc:398] Leaking SignalData structure 0x7b08000c05a0 after lost signal to thread 19630
I20260923 17:32:11.470695 19718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:11.471262 19718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:11.480340 19718 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
W20260923 17:32:11.500144 19580 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267): Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:32:11.504371 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:11.507557 19578 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd; no voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:11.513799 19718 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260923 17:32:12.333750 16346 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
/tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:35267
--webserver_interface=127.15.246.189
--webserver_port=36523
--builtin_ntp_servers=127.15.246.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--hive_metastore_uris=thrift://127.0.0.1:38515
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260923 17:32:12.724272 19733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 17:32:12.725275 19733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 17:32:12.726086 19733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 17:32:12.770243 19733 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260923 17:32:12.770527 19733 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260923 17:32:12.770761 19733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 17:32:12.771005 19733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 17:32:12.771232 19733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 17:32:12.771459 19733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 17:32:12.813808 19733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.246.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38515
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.246.189:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.246.189
--webserver_port=36523
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 16:47:15 UTC on ccycloud.quasar-ynfelf.root.comops.site
build id 4667753
TSAN enabled
I20260923 17:32:12.815436 19733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 17:32:12.817628 19733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 17:32:12.841933 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:13.046504 19754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:13.047282 19754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:13.049798 19754 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
W20260923 17:32:13.075142 19580 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267): Network error: Client connection negotiation failed: client connection to 127.15.246.189:35267: connect: Connection refused (error 111)
W20260923 17:32:13.075892 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:13.076646 19578 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469a88f1fddf4ecba558f342a2fe37fd; no voters: 05a13f8d28964563b4072f7542fd178b, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:13.077565 19754 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260923 17:32:12.842422 19748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:12.845299 19733 server_base.cc:1034] running on GCE node
W20260923 17:32:12.845044 19751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 17:32:14.022006 19733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 17:32:14.025202 19733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 17:32:14.026687 19733 hybrid_clock.cc:648] HybridClock initialized: now 1790184734026606 us; error 88 us; skew 500 ppm
I20260923 17:32:14.027561 19733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 17:32:14.050848 19733 webserver.cc:460] Webserver started at http://127.15.246.189:36523/ using document root <none> and password file <none>
I20260923 17:32:14.052007 19733 fs_manager.cc:359] Metadata directory not provided
I20260923 17:32:14.052289 19733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 17:32:14.061519 19733 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.001s
I20260923 17:32:14.068035 19777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 17:32:14.069633 19733 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260923 17:32:14.070014 19733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data,/tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
uuid: "05a13f8d28964563b4072f7542fd178b"
format_stamp: "Formatted at 2026-09-23 17:31:55 on dist-test-slave-mk2q"
I20260923 17:32:14.072757 19733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 17:32:14.115765 19733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 17:32:14.117357 19733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 17:32:14.117799 19733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 17:32:14.196786 19733 rpc_server.cc:307] RPC server started. Bound to: 127.15.246.189:35267
I20260923 17:32:14.196935 19828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.246.189:35267 every 8 connection(s)
I20260923 17:32:14.199679 19733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskMyBI3K/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790184654112867-16346-0/minicluster-data/master-1/data/info.pb
I20260923 17:32:14.205705 16346 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskMyBI3K/build/tsan/bin/kudu as pid 19733
I20260923 17:32:14.214803 19829 sys_catalog.cc:263] Verifying existing consensus state
I20260923 17:32:14.223670 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Bootstrap starting.
I20260923 17:32:14.250326 19829 log.cc:824] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Log is configured to *not* fsync() on all Append() calls
I20260923 17:32:14.283663 19111 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:14.327481 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=13 ignored=0} mutations{seen=17 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 17:32:14.328783 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: Bootstrap complete.
I20260923 17:32:14.359758 19829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:14.362849 19829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a13f8d28964563b4072f7542fd178b, State: Initialized, Role: FOLLOWER
I20260923 17:32:14.363916 19829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:14.367393 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:14.368405 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:14.368777 19829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 17:32:14.422320 19850 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:32:14.422672 19850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: loading cluster ID for follower catalog manager: success
I20260923 17:32:14.433286 19850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: acquiring CA information for follower catalog manager: success
I20260923 17:32:14.439283 19403 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:14.443280 19850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 17:32:14.406952 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:14.450312 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:14.452677 19794 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" instance_seqno: 1790184725330536) as {username='slave'} at 127.15.246.131:34145; Asking this server to re-register.
I20260923 17:32:14.454424 19403 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:14.458925 19794 ts_manager.cc:194] Registered new tserver with Master: 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485)
I20260923 17:32:14.513402 19471 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:14.513820 19471 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:14.704888 19258 heartbeater.cc:347] Connected to a master server at 127.15.246.189:35267
I20260923 17:32:14.705542 19258 heartbeater.cc:503] Master 127.15.246.189:35267 was elected leader, sending a full tablet report...
I20260923 17:32:14.710175 19794 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "2a823629310d45f880d1c553255a6d1b" instance_seqno: 1790184723449243) as {username='slave'} at 127.15.246.130:45383; Asking this server to re-register.
I20260923 17:32:14.711560 19258 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:14.715896 19794 ts_manager.cc:194] Registered new tserver with Master: 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
I20260923 17:32:14.751493 19857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:14.751873 19857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:14.756191 19857 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
W20260923 17:32:14.762646 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:14.766999 19804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b" is_pre_election: true
I20260923 17:32:14.767798 19804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 469a88f1fddf4ecba558f342a2fe37fd in term 3.
I20260923 17:32:14.768954 19580 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd; no voters: ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:14.769902 19857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260923 17:32:14.770288 19857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05a13f8d28964563b4072f7542fd178b)
I20260923 17:32:14.770608 19857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 3 FOLLOWER]: Advancing to term 4
I20260923 17:32:14.775341 19857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:14.779253 19857 leader_election.cc:288] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 election: Requested vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 05a13f8d28964563b4072f7542fd178b (127.15.246.189:35267)
I20260923 17:32:14.780834 19804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "469a88f1fddf4ecba558f342a2fe37fd" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: false dest_uuid: "05a13f8d28964563b4072f7542fd178b"
I20260923 17:32:14.781407 19804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 3 FOLLOWER]: Advancing to term 4
W20260923 17:32:14.783610 19578 leader_election.cc:334] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:14.788488 19804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 469a88f1fddf4ecba558f342a2fe37fd in term 4.
I20260923 17:32:14.791401 19580 leader_election.cc:302] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b, 469a88f1fddf4ecba558f342a2fe37fd; no voters: ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:14.792634 19857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 4 FOLLOWER]: Leader election won for term 4
I20260923 17:32:14.793607 19794 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" instance_seqno: 1790184721395473) as {username='slave'} at 127.15.246.129:37867; Asking this server to re-register.
I20260923 17:32:14.795534 19111 heartbeater.cc:464] Registering TS with master...
I20260923 17:32:14.796190 19857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [term 4 LEADER]: Becoming Leader. State: Replica: 469a88f1fddf4ecba558f342a2fe37fd, State: Running, Role: LEADER
I20260923 17:32:14.797257 19857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:14.798782 19794 ts_manager.cc:194] Registered new tserver with Master: dedc4fa600d8426c9074e3c8916aeaee (127.15.246.129:37787)
I20260923 17:32:14.803076 19860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a88f1fddf4ecba558f342a2fe37fd. Latest consensus state: current_term: 4 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:14.803645 19860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [sys.catalog]: This master's current role is: LEADER
I20260923 17:32:14.805259 19862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 17:32:14.813915 19862 catalog_manager.cc:669] Loaded metadata for table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a]
I20260923 17:32:14.815913 19862 catalog_manager.cc:669] Loaded metadata for table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db]
I20260923 17:32:14.817649 19862 catalog_manager.cc:669] Loaded metadata for table default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153]
I20260923 17:32:14.825519 19862 tablet_loader.cc:96] loaded metadata for tablet 6753a0b86b7646339cec60f5c47811f1 (table default.table_1da9836e55d14639a899e7d3f4d58b55 [id=85c52abfb3044f52b73e4ce6e203b153])
I20260923 17:32:14.832549 19862 tablet_loader.cc:96] loaded metadata for tablet a5900e799aca43bfa9b0dc1328bb6096 (table default.table_e6ca980ce29c4245bb88eaf5eb38907b [id=0d531f02e1834792b98bf7348fca884a])
I20260923 17:32:14.833632 19862 tablet_loader.cc:96] loaded metadata for tablet beb32e39cda941f1ae80851799093c49 (table default.table_78b2f53459124b318a200b9c57216ec9 [id=7d92bc7cedf94b75bfb904ad1c0468db])
I20260923 17:32:14.834674 19862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 17:32:14.838047 19862 catalog_manager.cc:1259] Loaded cluster ID: 0f1865b84a9846269b5ffd7cafa10c48
I20260923 17:32:14.838245 19862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 17:32:14.842180 19862 catalog_manager.cc:1504] Loading token signing keys...
I20260923 17:32:14.845564 19862 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd: Loaded TSK: 0
I20260923 17:32:14.846863 19862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 17:32:14.849977 19862 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260923 17:32:14.853068 19862 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
W20260923 17:32:14.881673 19594 catalog_manager.cc:5328] Ignoring report from unknown tablet b04c5431995d48c7b91f7bfad0ee90a6
I20260923 17:32:14.936036 19594 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:58320
I20260923 17:32:14.937288 19595 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58320:
name: "default.Table_6a36064c5aba4b21bd8b5da2ccb2212e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:14.938555 19593 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58320:
table { table_name: "default.Table_e6ca980ce29c4245bb88eaf5eb38907b" } new_table_name: "default.Table_fd2cd848c438499980de5e4bae0b10b6" modify_external_catalogs: true
I20260923 17:32:14.939029 19592 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58320:
name: "default.Table_0ba166e019864d25946c15765c3f0b71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:14.941017 19590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58320:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
I20260923 17:32:14.940864 19591 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58320:
name: "default.Table_af14483dad1b4c50a7033d85e9cbc651"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 17:32:14.941706 19590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58320:
table { table_name: "default.Table_1da9836e55d14639a899e7d3f4d58b55" } modify_external_catalogs: true
I20260923 17:32:14.943672 19589 master_service.cc:946] ReplaceTablet: received request to replace tablet 6440b97f09304b988f67124faacaee58 from {username='slave'} at 127.0.0.1:58320
I20260923 17:32:14.943629 19594 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58320:
table { table_name: "default.Table_78b2f53459124b318a200b9c57216ec9" } new_table_name: "default.Table_029ff3fda0d747e68a515ed3dd98d6e4" modify_external_catalogs: true
I20260923 17:32:14.945010 19588 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58320:
name: "default.Table_78ce2153804b4feab044ec3a58ea9f38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260923 17:32:14.947912 19650 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_319978fc115b4d10b8f8f3e8fac76165
*** Check failure stack trace: ***
@ 0x7feedfbd09a0 google::LogMessage::Flush() at ??:0
@ 0x7feedfbd070f google::LogMessage::~LogMessage() at ??:0
@ 0x7feeeac6baea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7feeeac6b33c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7feeeac6efd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7feeeac6ef8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
@ 0x7feeeac6ef1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7feeeac6eee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7feeeac6e1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
@ 0x7feee1d81dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260923 17:32:15.156836 19589 master_service.cc:946] ReplaceTablet: received request to replace tablet a5900e799aca43bfa9b0dc1328bb6096 from {username='slave'} at 127.0.0.1:58320
I20260923 17:32:15.160955 19587 master_service.cc:946] ReplaceTablet: received request to replace tablet b04c5431995d48c7b91f7bfad0ee90a6 from {username='slave'} at 127.0.0.1:58320
@ 0x7feee33ef6db start_thread at ??:0
I20260923 17:32:15.171703 19804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Refusing update from remote peer 469a88f1fddf4ecba558f342a2fe37fd: Log matching property violated. Preceding OpId in replica: term: 3 index: 23. Preceding OpId from leader: term: 4 index: 25. (index mismatch)
I20260923 17:32:15.173543 19860 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
W20260923 17:32:15.177081 19578 consensus_peers.cc:487] T 00000000000000000000000000000000 P 469a88f1fddf4ecba558f342a2fe37fd -> Peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Couldn't send request to peer ca93f1f37ccd46c08821589340bf34e1. Status: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7feede87561f clone at ??:0
I20260923 17:32:15.193157 19880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a88f1fddf4ecba558f342a2fe37fd. Latest consensus state: current_term: 4 leader_uuid: "469a88f1fddf4ecba558f342a2fe37fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } } }
I20260923 17:32:15.195108 19880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 17:32:15.383929 19110 heartbeater.cc:503] Master 127.15.246.188:41441 was elected leader, sending a full tablet report...
W20260923 17:32:15.429807 19778 connection.cc:535] server connection from 127.0.0.1:51814 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:15.430251 19143 connection.cc:535] client connection to 127.15.246.188:41441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:15.430687 19289 connection.cc:535] client connection to 127.15.246.188:41441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:15.431221 19412 connection.cc:535] client connection to 127.15.246.188:41441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:15.432466 18996 connection.cc:535] client connection to 127.15.246.188:41441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 17:32:15.433077 19412 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.15.246.188:41441)
W20260923 17:32:15.434247 19412 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.246.188:41441)
W20260923 17:32:15.434913 19412 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.15.246.188:41441)
W20260923 17:32:15.436383 19412 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.15.246.188:41441)
W20260923 17:32:15.440678 19887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 17:32:15.438096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.246.188:41441 (local address 127.0.0.1:58360)
0923 17:32:15.439259 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0923 17:32:15.439484 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":745,"threads_started":1}
I20260923 17:32:15.680932 19880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:15.681427 19880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:15.683974 19880 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:15.687479 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:15.691883 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:15.692517 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:15.692881 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:15.693639 19880 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:15.953356 19895 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:15.954272 19895 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
W20260923 17:32:15.960124 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:15.961091 19895 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:15.963753 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:15.964318 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:15.965590 19895 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:16.231547 19903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:16.232107 19903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:16.234112 19903 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:16.242723 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:16.243245 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:16.243626 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:16.244248 19903 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:16.633065 16346 master-stress-test.cc:578] Killing test thread 1/10
I20260923 17:32:17.333868 19914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:17.334348 19914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:17.336254 19914 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:17.341603 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:17.342164 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:17.342610 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:17.343278 19914 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:17.969326 19920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:17.969781 19920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:17.971459 19920 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:17.976331 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:17.976823 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:17.977180 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:17.977778 19920 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:32:18.790416 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 6: this message will repeat every 5th retry.
W20260923 17:32:19.162323 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 6: this message will repeat every 5th retry.
W20260923 17:32:19.177163 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000d55c0 after lost signal to thread 19123
W20260923 17:32:19.178077 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd4e0 after lost signal to thread 19255
I20260923 17:32:19.453339 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:19.453699 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:19.971083 19956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:19.971637 19956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:19.973757 19956 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:19.978971 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:32:19.979480 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:19.979856 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:19.980520 19956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:21.336613 19972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:21.337037 19972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:21.338575 19972 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:21.340611 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260923 17:32:21.343478 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:32:21.344022 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:21.344430 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:21.344985 19972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:24.457353 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:24.457831 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:32:25.326936 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000cb940 after lost signal to thread 18971
W20260923 17:32:25.327948 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000daf20 after lost signal to thread 19109
I20260923 17:32:25.452898 20029 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:25.453755 20029 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:25.456789 20029 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:25.459018 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:32:25.459619 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:25.459924 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:25.460619 20029 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:25.575452 20033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:25.575950 20033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:25.577562 20033 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:25.583118 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:25.583725 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:25.584019 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:25.584594 20033 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:32:28.530187 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 11: this message will repeat every 5th retry.
W20260923 17:32:28.582943 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 11: this message will repeat every 5th retry.
I20260923 17:32:28.775087 20075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:28.775452 20075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:28.777024 20075 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:28.780925 19778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260923 17:32:28.784204 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:32:28.784690 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:32:28.785115 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:28.785810 20075 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:29.460733 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:29.461148 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:30.170589 20098 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:30.171183 20098 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:30.172989 20098 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:30.174686 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:32:30.175186 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:30.175462 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:30.175997 20098 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:34.465092 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:34.465552 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:32:37.722044 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 16: this message will repeat every 5th retry.
W20260923 17:32:38.267323 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 16: this message will repeat every 5th retry.
I20260923 17:32:39.468381 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:39.468778 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:40.303984 20229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:40.304646 20229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:40.306501 20229 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:40.308490 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:32:40.311444 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:40.312034 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:40.312405 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:40.313020 20229 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:44.472379 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:44.472862 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:45.906951 20298 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:45.907493 20298 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:45.909725 20298 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:45.911608 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:32:45.912114 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:45.912501 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:45.913101 20298 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:32:47.595700 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 21: this message will repeat every 5th retry.
W20260923 17:32:48.125914 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 21: this message will repeat every 5th retry.
I20260923 17:32:48.470337 20338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:32:48.470878 20338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:32:48.472596 20338 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:32:48.474875 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:32:48.478061 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:32:48.478593 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:32:48.478915 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:32:48.479607 20338 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:32:49.476042 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:49.476438 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:32:52.536262 20394 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:32:52.536926 20394 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:32:52.538792 20394 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:32:52.540511 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:32:52.541069 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:32:52.541492 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:32:52.542037 20394 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:32:54.479660 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:54.480101 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:32:57.395262 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 26: this message will repeat every 5th retry.
W20260923 17:32:58.347625 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 26: this message will repeat every 5th retry.
I20260923 17:32:59.482985 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:32:59.483393 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:00.121819 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000b67a0 after lost signal to thread 19123
W20260923 17:33:00.122877 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8200 after lost signal to thread 19255
I20260923 17:33:01.361302 20515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:01.361841 20515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:01.363919 20515 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:01.366271 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:33:01.369179 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:33:01.369746 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:33:01.370144 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:01.370811 20515 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:04.298010 20563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:04.298727 20563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:04.300856 20563 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:04.306159 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:33:04.306747 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:33:04.307112 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:04.307783 20563 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:04.485999 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:04.486467 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:05.661751 19115 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (153 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:33:05.668179 19256 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (100 consecutive failures): Network error: Failed to ping master at 127.15.246.188:41441: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:33:06.290292 19404 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (155 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:33:07.200949 20594 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:07.201929 20594 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
W20260923 17:33:07.207378 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:07.208333 20594 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:07.213510 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 31: this message will repeat every 5th retry.
W20260923 17:33:07.215042 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:07.215579 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:07.216477 20594 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:33:08.970096 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 31: this message will repeat every 5th retry.
I20260923 17:33:09.172845 20617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:09.173228 20617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:09.174741 20617 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:09.177031 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260923 17:33:09.179715 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:33:09.180174 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:33:09.180584 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:09.181284 20617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:09.488830 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:09.489286 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:14.493069 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:14.493546 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:16.843741 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 36: this message will repeat every 5th retry.
I20260923 17:33:17.317251 20726 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:17.317812 20726 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:33:17.319669 20726 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:17.321439 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:33:17.321978 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:17.322309 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:17.322962 20726 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:33:18.812556 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 36: this message will repeat every 5th retry.
I20260923 17:33:19.496022 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:19.496444 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:23.363723 20800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:23.364430 20800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:23.366312 20800 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:23.368774 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:33:23.372699 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:33:23.373180 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:33:23.373493 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:23.374189 20800 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:24.499624 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:24.500204 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:26.126389 19825 debug-util.cc:398] Leaking SignalData structure 0x7b0800041240 after lost signal to thread 19737
W20260923 17:33:26.126938 19825 debug-util.cc:398] Leaking SignalData structure 0x7b080009bde0 after lost signal to thread 19828
W20260923 17:33:27.321367 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 41: this message will repeat every 5th retry.
W20260923 17:33:28.466768 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 41: this message will repeat every 5th retry.
I20260923 17:33:29.553649 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:29.554060 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:31.172312 20907 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:31.172889 20907 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:33:31.175133 20907 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:31.176829 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:33:31.177448 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:31.177770 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:31.178369 20907 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:33:34.557900 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:34.558429 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:35.023949 20947 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:35.024503 20947 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:33:35.026260 20947 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:35.028191 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:33:35.028703 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:35.029047 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:35.029681 20947 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:33:36.747170 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 46: this message will repeat every 5th retry.
W20260923 17:33:38.516817 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 46: this message will repeat every 5th retry.
W20260923 17:33:39.297317 19106 debug-util.cc:398] Leaking SignalData structure 0x7b0800040440 after lost signal to thread 18971
W20260923 17:33:39.298393 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9f60 after lost signal to thread 19109
I20260923 17:33:39.561437 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:39.561883 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:39.892602 21018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:39.893121 21018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:39.895030 21018 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:39.896325 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:33:39.899281 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:33:39.899864 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:33:39.900128 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:39.900681 21018 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:44.565577 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:44.566048 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:46.765821 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 51: this message will repeat every 5th retry.
I20260923 17:33:47.096387 21111 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:47.096942 21111 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:33:47.098906 21111 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:47.100878 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:33:47.101385 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:47.101670 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:47.102175 21111 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:33:48.277747 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 51: this message will repeat every 5th retry.
I20260923 17:33:49.529953 21141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:49.530454 21141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:49.532110 21141 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:49.534310 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:33:49.537168 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:33:49.537659 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:33:49.538012 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:49.538573 21141 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:49.568485 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:49.568917 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:33:52.585880 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3f40 after lost signal to thread 19268
W20260923 17:33:52.586982 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000e1600 after lost signal to thread 19401
I20260923 17:33:54.572849 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:54.573290 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:33:56.070026 21228 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:33:56.070546 21228 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:33:56.072626 21228 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:33:56.075426 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:33:56.076368 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:33:56.076738 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:33:56.078226 21228 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:33:57.295828 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 56: this message will repeat every 5th retry.
W20260923 17:33:57.898358 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 56: this message will repeat every 5th retry.
I20260923 17:33:58.929196 21267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:33:58.929756 21267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:33:58.931460 21267 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:33:58.933542 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:33:58.936722 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:33:58.937286 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:33:58.937556 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:33:58.938088 21267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:33:59.576566 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:33:59.577013 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:34:01.350163 21301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:34:01.350639 21301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:34:01.352342 21301 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:34:01.357599 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:01.358006 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:34:01.358319 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:34:01.358884 21301 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:34:04.393952 21344 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:34:04.394568 21344 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:34:04.396993 21344 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:34:04.398957 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:34:04.399437 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:34:04.399760 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:34:04.400372 21344 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:34:04.581284 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:04.581792 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:05.695873 19259 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (241 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:05.835604 19110 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (272 consecutive failures): Network error: Failed to ping master at 127.15.246.188:41441: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:34:06.661739 19404 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (304 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:07.320209 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 61: this message will repeat every 5th retry.
W20260923 17:34:07.566709 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 61: this message will repeat every 5th retry.
I20260923 17:34:09.588532 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:09.589111 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:34:14.456187 19783 maintenance_manager.cc:419] P 05a13f8d28964563b4072f7542fd178b: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033358
I20260923 17:34:14.538437 19782 maintenance_manager.cc:643] P 05a13f8d28964563b4072f7542fd178b: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.081s user 0.042s sys 0.013s Metrics: {"bytes_written":9211,"cfile_init":1,"drs_written":1,"lbm_read_time_us":213,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2303,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":13,"thread_start_us":560,"threads_started":1,"wal-append.queue_time_us":922}
I20260923 17:34:14.541617 19783 maintenance_manager.cc:419] P 05a13f8d28964563b4072f7542fd178b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 827 bytes on disk
I20260923 17:34:14.543541 19782 maintenance_manager.cc:643] P 05a13f8d28964563b4072f7542fd178b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":320,"lbm_reads_lt_1ms":4}
I20260923 17:34:14.592540 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:14.593008 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:17.076814 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 66: this message will repeat every 5th retry.
W20260923 17:34:17.623123 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 66: this message will repeat every 5th retry.
I20260923 17:34:18.644306 21528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:34:18.644820 21528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:34:18.646664 21528 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:34:18.648962 19778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260923 17:34:18.651808 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:18.652361 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:34:18.652742 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:34:18.653352 21528 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:34:19.596271 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:19.596733 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:34:19.644892 21534 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:34:19.645552 21534 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:34:19.648020 21534 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:34:19.650069 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:34:19.650638 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:34:19.650964 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:34:19.651536 21534 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:34:22.801663 19252 debug-util.cc:398] Leaking SignalData structure 0x7b0800085900 after lost signal to thread 19123
W20260923 17:34:22.802822 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2920 after lost signal to thread 19255
I20260923 17:34:23.713726 21595 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:34:23.714308 21595 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:34:23.716660 21595 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:34:23.718027 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:34:23.718775 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:34:23.719105 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:34:23.719643 21595 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:34:24.600811 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:24.601339 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:27.210067 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 71: this message will repeat every 5th retry.
W20260923 17:34:27.783926 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 71: this message will repeat every 5th retry.
W20260923 17:34:28.924350 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000ddb80 after lost signal to thread 19123
W20260923 17:34:28.925377 19252 debug-util.cc:398] Leaking SignalData structure 0x7b0800085540 after lost signal to thread 19255
I20260923 17:34:29.604425 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:29.604830 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:32.489753 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000c66e0 after lost signal to thread 19268
W20260923 17:34:32.490653 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9e80 after lost signal to thread 19401
I20260923 17:34:34.608563 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:34.609000 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:34:34.760277 21784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:34:34.760774 21784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:34:34.762703 21784 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:34:34.764104 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:34:34.767086 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:34.767603 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:34:34.767894 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:34:34.768496 21784 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:34:36.444808 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 76: this message will repeat every 5th retry.
W20260923 17:34:37.886765 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 76: this message will repeat every 5th retry.
I20260923 17:34:39.612190 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:39.612679 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:34:41.964610 21908 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:34:41.965155 21908 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:34:41.967546 21908 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:34:41.969597 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:34:41.970062 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:34:41.970315 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:34:41.970858 21908 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:34:43.449396 21933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:34:43.450063 21933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:34:43.452028 21933 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:34:43.455114 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:43.455142 19778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:34:43.458536 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:34:43.459009 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:34:43.459312 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:34:43.459944 21933 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:34:44.615782 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:44.616210 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:46.197472 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 81: this message will repeat every 5th retry.
W20260923 17:34:47.591020 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 81: this message will repeat every 5th retry.
I20260923 17:34:49.619271 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:49.619774 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:53.086500 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdca0 after lost signal to thread 19123
W20260923 17:34:53.087745 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdf60 after lost signal to thread 19255
I20260923 17:34:54.623803 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:54.624469 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:34:56.310933 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 86: this message will repeat every 5th retry.
W20260923 17:34:57.691043 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 86: this message will repeat every 5th retry.
I20260923 17:34:59.627626 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:34:59.628032 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:35:00.576092 22203 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:00.576781 22203 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:00.579348 22203 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:00.581403 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:00.581924 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:00.582198 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:00.582870 22203 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:35:02.981809 22248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:35:02.982376 22248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:35:02.984563 22248 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:35:02.986709 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:02.999487 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:35:03.005935 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:35:03.006340 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:35:03.007601 22248 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:35:04.632483 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:04.632989 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:05.997452 19259 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (361 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:06.220191 19115 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (447 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:06.883301 19402 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (427 consecutive failures): Network error: Failed to ping master at 127.15.246.188:41441: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:35:07.101217 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 91: this message will repeat every 5th retry.
W20260923 17:35:07.569000 19825 debug-util.cc:398] Leaking SignalData structure 0x7b08000be880 after lost signal to thread 19737
W20260923 17:35:07.569747 19825 debug-util.cc:398] Leaking SignalData structure 0x7b08000505a0 after lost signal to thread 19828
W20260923 17:35:08.257918 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 91: this message will repeat every 5th retry.
I20260923 17:35:09.670778 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:09.671159 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:35:14.674839 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:14.675258 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:35:16.306651 22470 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:16.307433 22470 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:16.310633 22470 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:16.311913 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:16.312637 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:16.313060 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:16.313638 22470 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:35:17.111322 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 96: this message will repeat every 5th retry.
W20260923 17:35:19.111048 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 96: this message will repeat every 5th retry.
I20260923 17:35:19.678602 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:19.679039 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:35:21.615613 22555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:35:21.616206 22555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:35:21.618252 22555 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:35:21.620481 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:35:21.623485 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:21.624083 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:35:21.624465 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:35:21.625072 22555 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:35:24.361936 22595 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:24.362881 22595 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:24.365201 22595 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:24.367013 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:24.367517 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:24.367760 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:24.368335 22595 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:35:24.683019 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:24.683543 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:26.261879 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 101: this message will repeat every 5th retry.
I20260923 17:35:29.686713 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:29.687057 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:29.759198 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 101: this message will repeat every 5th retry.
I20260923 17:35:31.724467 22714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:35:31.725055 22714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:35:31.726927 22714 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:35:31.729183 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:35:31.732385 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:35:31.732889 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:35:31.733311 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:35:31.733999 22714 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:35:31.873728 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6f60 after lost signal to thread 18971
W20260923 17:35:31.874832 19106 debug-util.cc:398] Leaking SignalData structure 0x7b080006fa40 after lost signal to thread 19109
I20260923 17:35:34.690623 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:34.691236 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:35.719218 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 106: this message will repeat every 5th retry.
W20260923 17:35:38.242714 19825 debug-util.cc:398] Leaking SignalData structure 0x7b080009d3c0 after lost signal to thread 19737
W20260923 17:35:38.243485 19825 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad720 after lost signal to thread 19828
I20260923 17:35:38.746954 22833 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:38.747511 22833 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:38.749713 22833 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:38.751435 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:38.751850 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:38.752104 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:38.752714 22833 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:35:39.190374 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 106: this message will repeat every 5th retry.
I20260923 17:35:39.706492 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:39.706900 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:43.334995 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000c66c0 after lost signal to thread 19268
W20260923 17:35:43.335839 19398 debug-util.cc:398] Leaking SignalData structure 0x7b0800040580 after lost signal to thread 19401
I20260923 17:35:44.176482 22906 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:44.177151 22906 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:44.178982 22906 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:44.180833 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:44.181382 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:44.181632 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:44.182221 22906 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:35:44.710784 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:44.711305 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:46.136667 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 111: this message will repeat every 5th retry.
W20260923 17:35:48.660651 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 111: this message will repeat every 5th retry.
W20260923 17:35:49.482149 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3180 after lost signal to thread 18971
W20260923 17:35:49.483065 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6e60 after lost signal to thread 19109
I20260923 17:35:49.714129 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:49.714463 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:35:50.521303 23011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:35:50.521863 23011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:35:50.523941 23011 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:35:50.526520 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:35:50.530790 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:50.531770 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:35:50.532191 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:35:50.533079 23011 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:35:52.012676 23041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:35:52.013168 23041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:35:52.014931 23041 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:35:52.020305 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:35:52.020840 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:35:52.021162 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:35:52.021749 23041 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:35:54.718277 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:54.718767 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:35:56.577520 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 116: this message will repeat every 5th retry.
I20260923 17:35:56.892943 23118 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:35:56.893476 23118 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:35:56.895529 23118 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:35:56.897454 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:35:56.897972 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:35:56.898314 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:35:56.898917 23118 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:35:58.316416 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 116: this message will repeat every 5th retry.
I20260923 17:35:59.721496 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:35:59.721877 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:00.337225 23171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:00.337771 23171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:00.339497 23171 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:00.341543 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260923 17:36:00.344895 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:36:00.345404 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:36:00.345750 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:00.346371 23171 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:36:04.725535 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:04.725996 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:36:06.266783 19259 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (481 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:36:06.448099 19115 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (596 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:36:07.157335 19402 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (577 consecutive failures): Network error: Failed to ping master at 127.15.246.188:41441: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:36:07.348014 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 121: this message will repeat every 5th retry.
W20260923 17:36:08.213371 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 121: this message will repeat every 5th retry.
I20260923 17:36:09.729038 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:09.729545 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:12.025180 23341 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:36:12.025813 23341 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:36:12.028375 23341 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:36:12.030073 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:36:12.030637 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:36:12.030948 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:36:12.031551 23341 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:36:14.733335 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:14.733793 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:36:16.358664 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 126: this message will repeat every 5th retry.
I20260923 17:36:16.501703 23409 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:36:16.502174 23409 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:36:16.504177 23409 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:36:16.506335 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:36:16.506836 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:36:16.507218 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:36:16.507817 23409 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:36:18.312850 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 126: this message will repeat every 5th retry.
I20260923 17:36:18.582902 23439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:18.583487 23439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:18.586098 23439 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:18.588861 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:36:18.591714 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:36:18.592792 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:36:18.593089 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:18.593653 23439 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:36:19.733575 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2140 after lost signal to thread 18971
W20260923 17:36:19.734937 19106 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5e80 after lost signal to thread 19109
I20260923 17:36:19.737318 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:19.737748 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:24.741921 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:24.742435 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:36:26.780720 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 131: this message will repeat every 5th retry.
I20260923 17:36:26.945259 23583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:26.945860 23583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:26.947595 23583 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:26.949524 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:36:26.952944 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:36:26.953469 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:36:26.953758 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:26.954324 23583 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:36:28.019111 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 131: this message will repeat every 5th retry.
I20260923 17:36:29.745603 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:29.745975 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:34.749691 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:34.750178 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:34.847532 23704 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:36:34.848248 23704 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:36:34.850533 23704 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:36:34.852636 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:36:34.853258 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:36:34.853600 18997 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:36:34.854355 23704 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:36:35.671746 23722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:35.672269 23722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:35.673888 23722 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:35.675935 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:36:35.678566 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:36:35.679080 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:36:35.679378 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:35.679955 23722 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260923 17:36:36.366154 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6900 after lost signal to thread 19268
W20260923 17:36:36.367049 19398 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2040 after lost signal to thread 19401
W20260923 17:36:36.454505 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 136: this message will repeat every 5th retry.
W20260923 17:36:36.737224 19252 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdda0 after lost signal to thread 19123
W20260923 17:36:36.738552 19252 debug-util.cc:398] Leaking SignalData structure 0x7b0800043b20 after lost signal to thread 19255
W20260923 17:36:37.550434 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 136: this message will repeat every 5th retry.
I20260923 17:36:39.753664 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:39.754101 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:44.762223 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:44.762722 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:36:45.730572 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 141: this message will repeat every 5th retry.
W20260923 17:36:48.474931 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 141: this message will repeat every 5th retry.
I20260923 17:36:49.766372 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:49.766813 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:36:50.378500 23956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:50.379004 23956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:50.380800 23956 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:50.383972 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260923 17:36:50.386240 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:36:50.386803 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:36:50.387053 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:50.387673 23956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:36:53.699844 24012 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:36:53.700376 24012 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:36:53.702593 24012 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:36:53.704133 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:36:53.704807 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:36:53.705251 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:36:53.705897 24012 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 17:36:53.733605 24013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:36:53.734256 24013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:36:53.736291 24013 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:36:53.741732 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:36:53.742352 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:36:53.742679 19778 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:36:53.743279 24013 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:36:54.770102 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:54.770582 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:36:55.751173 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 146: this message will repeat every 5th retry.
W20260923 17:36:57.831097 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 146: this message will repeat every 5th retry.
W20260923 17:36:58.904191 19825 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3680 after lost signal to thread 19737
W20260923 17:36:58.904733 19825 debug-util.cc:398] Leaking SignalData structure 0x7b080008b220 after lost signal to thread 19828
I20260923 17:36:59.800675 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:36:59.801200 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 17:37:04.804692 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:37:04.805163 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:37:04.984946 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 151: this message will repeat every 5th retry.
W20260923 17:37:06.015646 19825 debug-util.cc:398] Leaking SignalData structure 0x7b080009a4e0 after lost signal to thread 19737
W20260923 17:37:06.016182 19825 debug-util.cc:398] Leaking SignalData structure 0x7b08000af1e0 after lost signal to thread 19828
W20260923 17:37:06.523334 19259 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (601 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
W20260923 17:37:06.787299 19115 heartbeater.cc:644] Failed to heartbeat to 127.15.246.190:46265 (746 consecutive failures): Network error: Failed to ping master at 127.15.246.190:46265: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:37:06.954207 24212 raft_consensus.cc:482] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 17:37:06.954723 24212 raft_consensus.cc:504] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14ff7c89a64a1d9b877b5bfbd443cd" member_type: VOTER last_known_addr { host: "127.15.246.131" port: 34485 } } peers { permanent_uuid: "dedc4fa600d8426c9074e3c8916aeaee" member_type: VOTER last_known_addr { host: "127.15.246.129" port: 37787 } } peers { permanent_uuid: "2a823629310d45f880d1c553255a6d1b" member_type: VOTER last_known_addr { host: "127.15.246.130" port: 43119 } }
I20260923 17:37:06.956377 24212 leader_election.cc:288] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485), 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119)
W20260923 17:37:06.958221 18997 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14ff7c89a64a1d9b877b5bfbd443cd (127.15.246.131:34485): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
W20260923 17:37:06.958820 18996 leader_election.cc:341] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Not found: Tablet not found: 6440b97f09304b988f67124faacaee58
I20260923 17:37:06.959096 18996 leader_election.cc:302] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dedc4fa600d8426c9074e3c8916aeaee; no voters: 2a823629310d45f880d1c553255a6d1b, 7f14ff7c89a64a1d9b877b5bfbd443cd
I20260923 17:37:06.959686 24212 raft_consensus.cc:2726] T 6440b97f09304b988f67124faacaee58 P dedc4fa600d8426c9074e3c8916aeaee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 17:37:07.484242 18996 consensus_peers.cc:487] T b04c5431995d48c7b91f7bfad0ee90a6 P dedc4fa600d8426c9074e3c8916aeaee -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b04c5431995d48c7b91f7bfad0ee90a6. This is attempt 151: this message will repeat every 5th retry.
W20260923 17:37:07.562640 19402 heartbeater.cc:644] Failed to heartbeat to 127.15.246.188:41441 (729 consecutive failures): Network error: Failed to ping master at 127.15.246.188:41441: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
I20260923 17:37:09.700970 24258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 469a88f1fddf4ecba558f342a2fe37fd)
I20260923 17:37:09.701476 24258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca93f1f37ccd46c08821589340bf34e1" member_type: VOTER last_known_addr { host: "127.15.246.190" port: 46265 } } peers { permanent_uuid: "05a13f8d28964563b4072f7542fd178b" member_type: VOTER last_known_addr { host: "127.15.246.189" port: 35267 } } peers { permanent_uuid: "469a88f1fddf4ecba558f342a2fe37fd" member_type: VOTER last_known_addr { host: "127.15.246.188" port: 41441 } }
I20260923 17:37:09.703830 24258 leader_election.cc:288] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265), 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441)
W20260923 17:37:09.706139 19779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260923 17:37:09.708755 19778 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 469a88f1fddf4ecba558f342a2fe37fd (127.15.246.188:41441): Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection refused (error 111)
W20260923 17:37:09.709282 19779 leader_election.cc:334] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ca93f1f37ccd46c08821589340bf34e1 (127.15.246.190:46265): Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111)
I20260923 17:37:09.709615 19779 leader_election.cc:302] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a13f8d28964563b4072f7542fd178b; no voters: 469a88f1fddf4ecba558f342a2fe37fd, ca93f1f37ccd46c08821589340bf34e1
I20260923 17:37:09.710171 24258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 05a13f8d28964563b4072f7542fd178b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260923 17:37:09.771875 19783 maintenance_manager.cc:419] P 05a13f8d28964563b4072f7542fd178b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 827 bytes on disk
I20260923 17:37:09.790035 19782 maintenance_manager.cc:643] P 05a13f8d28964563b4072f7542fd178b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.011s sys 0.005s Metrics: {"dirs.queue_time_us":1965,"dirs.run_cpu_time_us":279,"dirs.run_wall_time_us":280,"thread_start_us":2469,"threads_started":2,"wal-append.queue_time_us":1313}
I20260923 17:37:09.840688 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:37:09.841058 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:37:14.723198 19289 consensus_peers.cc:487] T bb70d5dd96c44b9cac045d923f6d825c P 7f14ff7c89a64a1d9b877b5bfbd443cd -> Peer 2a823629310d45f880d1c553255a6d1b (127.15.246.130:43119): Couldn't send request to peer 2a823629310d45f880d1c553255a6d1b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb70d5dd96c44b9cac045d923f6d825c. This is attempt 156: this message will repeat every 5th retry.
I20260923 17:37:14.844748 19733 master_runner.cc:186] Error getting master registration for 127.15.246.190:46265: Network error: Client connection negotiation failed: client connection to 127.15.246.190:46265: connect: Connection refused (error 111),
I20260923 17:37:14.845242 19733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 17:37:15.463379 19412 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
W20260923 17:37:15.463900 19412 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
W20260923 17:37:15.464138 19412 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
W20260923 17:37:15.464497 19412 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
F20260923 17:37:15.476452 19424 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20260923 17:37:15.476490 19421 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0ba166e019864d25946c15765c3f0b71 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
*** Aborted at 1790185035 (unix time) try "date -d @1790185035" if you are using GNU date ***
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800003fda) received by PID 16346 (TID 0x7fbe9e2f9700) from PID 16346; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7fbeb05a1980 (unknown) at ??:0
@ 0x7fbeab85ee87 gsignal at ??:0
@ 0x7fbeab8607f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7fbeae892df4 google::LogMessage::SendToLog() at ??:0
@ 0x7fbeae8939a0 google::LogMessage::Flush() at ??:0
@ 0x7fbeae898adb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x3578b6 kudu::MasterStressTest::AlterTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
F20260923 17:37:15.476490 19421 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0ba166e019864d25946c15765c3f0b71 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)F000F20260923 17:37:15.485939 19425 master-stress-test.cc:392] 000F20260923 17:37:15.491994 19428 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.15.246.190:46265,127.15.246.189:35267,127.15.246.188:41441, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.15.246.188:41441: connect: Connection reset by peer (error 104)
*** Check failure stack trace: ***
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7fbeb05966db start_thread at ??:0
@ 0x7fbeab94161f clone at ??:0